Skip to content

AI & Technologyllms.txt: What It Does and Doesn't Do (2026)

Simon Dziak
Simon Dziak
Founder & developer, App369
LinkedIn
October 20, 2026Facts last verified October 20, 20267 min

Key takeaways

  • llms.txt is "A proposal to standardise on using an /llms.txt file to provide information to help agents use a website." It is not an official web standard.
  • Its own author says the files "are used most heavily for software documentation, where coding agents follow them to find API references and tutorials."
  • Google says an llms.txt file "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them."
  • None of the crawler pages from OpenAI, Anthropic or Perplexity says their products read other sites' llms.txt files to choose what to cite.
  • llms.txt doesn't control access. To allow or block AI crawlers, use robots.txt.

Pricing an app? Try the free app cost calculator or get a range for your project in the instant quote.

llms.txt is a Markdown file at the root of a website that gives AI tools a short summary and a list of links to the site's most useful pages. It helps coding agents find documentation, but Google says it does nothing for Search rankings, and OpenAI, Anthropic and Perplexity don't document using it to pick what to cite. That's the short version. The rest of this guide shows where each part of that comes from, so you can decide whether it's worth an hour of your time.

I re-read each source on October 20, 2026. This site publishes its own llms.txt, for the reason in the last section.

What llms.txt is

The proposal at llmstxt.org, written by Jeremy Howard, describes itself as "A proposal to standardise on using an /llms.txt file to provide information to help agents use a website." It was first published on September 3, 2024 and is now in a second version, modified on August 10, 2026.

The idea is simple: web pages wrap their content in navigation, ads and scripts, so give AI tools a clean starting point. The file lives "at the root path /llms.txt of a website or at any subpath," and the format is:

  • An H1 with the name of the project or site, which the proposal says "is the only required section".
  • A short summary in a blockquote.
  • Sections with lists of links, ideally to clean Markdown versions of your pages.

The proposal also suggests offering Markdown versions of pages and linking them from your HTML.

Who actually uses it

The proposal's own author is clear about where it's used: "llms.txt files are used most heavily for software documentation, where coding agents follow them to find API references and tutorials."

The proposal also says "thousands of sites publish an llms.txt file," and that the AI labs publish llms.txt files for their own developer docs, naming OpenAI, Anthropic and Gemini. That's true and easy to check (OpenAI's and Perplexity's docs both point to one), but publishing a file is not the same as reading everyone else's. It tells you the labs want coding tools to find their own docs.

What it doesn't do

It doesn't help in Google. Google's AI optimization guide says: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them." It adds that publishing them "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." In its summary, Google lists "creating unnecessary AI text files (like llms.txt)" among tactics you can ignore.

There's no documented citation benefit elsewhere. OpenAI's, Perplexity's and Anthropic's crawler pages explain how their search and training crawlers work and how to control them with robots.txt. None of them says their crawlers or products read other sites' llms.txt files to choose sources. I'm careful with the wording here: that's an absence of evidence, not a statement from those companies that they ignore it. But if someone sells llms.txt as a way to get cited by ChatGPT, ask them for the source.

It doesn't control access. llms.txt is a guide, not a permission file. If you want ChatGPT, Perplexity or Claude to be able to find you in search, allow their search crawlers in robots.txt. See how to get your SaaS cited by ChatGPT and Perplexity for the exact crawler names.

When it's worth doing

  • You have developer docs or an API. Coding tools are its main audience. A good llms.txt with links to clean Markdown docs makes your product easier to build with.
  • Your docs platform makes one automatically. Then it costs nothing. Check it's accurate.
  • A marketing site only. It's optional. It won't hurt, but don't expect traffic or citations from it, and don't let it replace the basics: indexable pages, the right crawlers allowed, and clear, sourced answers.

How to write one in under an hour

# Your Product

> Your Product is invoicing software for freelancers in the US and Canada. Plans start at $X a month.

## Docs
- [Getting started](https://yourdomain.com/docs/start.md): set up an account and send a first invoice
- [API reference](https://yourdomain.com/docs/api.md): endpoints, authentication and rate limits

## Product
- [Pricing](https://yourdomain.com/pricing): plans, limits and what each includes
- [Integrations](https://yourdomain.com/integrations): Stripe, QuickBooks and Zapier

Then check that:

  1. yourdomain.com/llms.txt returns a 200 as plain text or Markdown.
  2. Every link works and uses the full URL.
  3. The facts in it match your pricing and feature pages. An out-of-date llms.txt is worse than none.
  4. Your server logs show who fetches it. That's the only real way to know whether anything reads it.

Why this site has one

App369 publishes an llms.txt because it's cheap to keep current and gives coding and research agents a clean list of the guides and services here. I don't count it as an SEO or citation tactic, and nothing in this guide depends on it.

For how AI search fits with SEO, see AEO vs GEO vs SEO.

Need a hand?

If you'd rather have your site checked for AI search, see SEO, GEO and AEO services. The AI Search Visibility Audit is a $750 fixed price: I run your buyers' questions through ChatGPT, Perplexity, Google AI Overviews and Gemini, record who gets cited, and rank the fixes. Or send me a short description.

Frequently asked questions

Does llms.txt help SEO?
Not in Google. Google says it neither harms nor helps your visibility or rankings, because Google Search ignores these files. Its AI optimization guide lists llms.txt among tactics you can ignore for Google Search.
Does ChatGPT or Perplexity use llms.txt?
Their crawler documentation doesn't say so. OpenAI and Perplexity explain search and training through robots.txt and their crawlers. Their docs sites publish an llms.txt for their own documentation, but neither says it reads yours to choose sources. That isn't proof they never read it, but there is no documented benefit for citations.
Is llms.txt like robots.txt?
No. robots.txt tells crawlers what they may fetch. llms.txt is a guide that points AI tools to your most useful content. It doesn't allow or block anything.
Should my SaaS have an llms.txt file?
If you have developer docs or an API, yes: coding tools are its main audience and it takes under an hour. For a marketing site alone, it's optional and the benefit is unmeasured. It won't hurt, but don't expect it to bring traffic.
Where does llms.txt go?
At the root of your site, at /llms.txt, though the proposal also allows a subpath. It is a Markdown file whose only required part is an H1 with the name of the project or site.

Sources

  1. llmstxt.org, The /llms.txt file (v2, modified August 10, 2026)
  2. Google Search Central, Optimizing for generative AI search (updated 2026-07-10)
  3. Google Search Central, AI features and your website
  4. OpenAI, Overview of OpenAI crawlers
  5. Perplexity, Perplexity crawlers
  6. Anthropic, Does Anthropic crawl data from the web?
Tags
#llms.txt #what is llms.txt #does llms.txt help seo #llms.txt google #llms.txt chatgpt #ai crawlers #robots.txt vs llms.txt #generative engine optimization
Share:

Related Resources

Related guides