An llms.txt file is a plain-text index, written in markdown and placed at yoursite.com/llms.txt, that tells AI tools and coding agents what your site is about and where the important content lives. It doesn't get you cited in ChatGPT or ranked in Perplexity: Google's own 2026 search guidance says plainly that Search doesn't use it, and a study of 137,000 domains found 97% of these files are never read by anything at all.

What an llms.txt file is

Jeremy Howard, co-founder of Answer.AI and fast.ai, proposed the format in September 2024. The idea: large language models have small context windows and a hard time parsing a full HTML page with its navigation, ads, and scripts, so a site offers a clean, curated markdown summary instead.

The spec is simple. The file lives at the site's root, not a subfolder. It's markdown, not XML or JSON. It opens with an H1 for the project or site name (the only required section), a blockquote with a short summary, then optional sections and H2-delimited lists of links with brief descriptions. Here's a stripped-down example:

# Your Site Name

> One or two sentences on what this site or business does.

## Docs

- [Getting started](https://yoursite.com/docs/start.md): Setup and first steps
- [API reference](https://yoursite.com/docs/api.md): Full endpoint list

## Optional

- [Extended background](https://yoursite.com/docs/background.md)

Where the "it gets you found by AI" idea came from

Howard's original proposal was aimed at developers: help an AI coding assistant find a library's documentation without crawling the whole site. Ahrefs, which ran the largest adoption study to date, put it plainly: "the 'AI visibility' framing around llms.txt came later on, attached by the SEO industry as adoption spread on the speculation that AI platforms would reward the file." The name sounds like robots.txt or sitemap.xml, so it's an easy leap to assume it works the same way for AI search. Nothing in the spec or in any AI vendor's public documentation backs that leap up.

What the 2026 data shows

Google's Search Central AI optimization guide, updated June 15, 2026, states directly that you don't need machine-readable AI files, markup, or markdown to appear in Google Search or its generative features, because Search doesn't use them. Google's John Mueller has compared the file to the old keywords meta tag: a site owner's unverified claim about what the site is about, which search engines learned to distrust decades ago.

Ahrefs analyzed 137,210 domains in May 2026. 28% had published an llms.txt file. Of those, 97% received zero requests, from anything, for the entire month. Of the 3% that did get read, only 19.5% of those requests came from named AI tools, and the biggest single readers were GPTBot (which trains models, it doesn't answer live queries) and Claude Code (a coding agent, not a search engine). AI retrieval bots like PerplexityBot and OAI-SearchBot, the ones that answer questions in AI search products, made up just 1.1% of all requests. A separate 90-day study from Otterly.AI found the same pattern: 84 llms.txt requests out of 62,100 total AI bot visits, about 0.1%.

Who reads it

The file's real audience is coding agents and developer tooling: Claude Code, Cursor, GitHub Copilot, Windsurf, and similar tools look for /llms.txt as a shortcut when a developer points them at a product's documentation. Chrome's Lighthouse added an llms.txt check to its "Agentic Browsing" audits in May 2026, which is a different bet than Google Search's own guidance to skip it. That split is real: Chrome is optimizing for agents that browse on a developer's behalf, and Search is optimizing for the ranking algorithm, and right now those are two different audiences reading two different signals.

Should you still make one?

Probably, but for the right reason. An llms.txt file won't move your ranking in Google, won't get you cited more often in ChatGPT or Perplexity, and won't replace the work of writing clear, well-structured, crawlable content, which is still what earns AI citations. It's cheap to build (a page like this takes under an hour), it doesn't carry any downside, and if you publish documentation or a product that developers work with through AI coding tools, it's a real, if small, convenience for that audience.

Skip it if your goal is showing up more in AI search answers for a general audience. Spend that time on content that answers real questions clearly instead. That's what the retrieval bots feeding AI search answers are reading.

How to create a basic llms.txt file

  1. Write it in plain markdown and save it as llms.txt.
  2. Put it at your site's root, so it loads at yoursite.com/llms.txt, not a subfolder.
  3. Start with an H1 for your site or project name, then a blockquote with a one- or two-sentence summary.
  4. Add H2 sections that list your most useful pages as markdown links, each with a short, honest description.
  5. Keep it current. A stale file misleads any agent that trusts it, and treat it like code: version it and review changes before publishing.

The short version

An llms.txt file is a markdown index at your site's root that helps AI tools and coding agents understand your site faster. It doesn't improve your AI search visibility today: Google says Search ignores it, and independent studies put real readership at 3% of published files or less, mostly from bots that aren't consumer AI search at all. Build one if developers use AI coding tools against your docs. Don't build one expecting a ranking or citation boost.

Want more explainers that cut through the AI hype? Browse the rest of the blog, or if you've got a tech question you'd like answered, send it my way.

Sources