llms.txt explained, what it is and whether you need one

llms.txt is a proposed standard for a plain-text Markdown file at the root of a website, such as yoursite.com/llms.txt, that gives AI systems a short summary of the site and links to its most useful pages. It is cheap to add, but no major AI search engine has confirmed it uses llms.txt when deciding what to show or cite.

Where llms.txt came from

The proposal was published in September 2024 by Jeremy Howard of Answer.AI. The idea is simple: large language models have limited context, and websites are full of navigation, scripts and layout that make pages harder for a model to read. A short, curated Markdown file could point models straight to the content that matters.

The proposal describes it as something like a guide for AI tools, distinct from robots.txt (which tells crawlers what they may access) and XML sitemaps (which list pages for indexing).

What the file looks like

The proposed format is Markdown with a light structure:

For a B2B service company, a hypothetical file might have a heading with the company name, a summary sentence saying what the company does and for whom, then sections such as Services, Pricing, Guides and Company, each listing a few key URLs with one-line descriptions.

The proposal also suggests offering Markdown versions of important pages (for example, page.html.md). Some documentation platforms also generate an llms-full.txt file containing the full text of the documentation in one file. That second file is a convention that grew around the proposal, not a requirement of it.

How llms.txt compares with robots.txt and sitemaps

llms.txt does not control anything. It cannot block or allow crawlers. If you want to manage AI crawler access, that happens in robots.txt and firewall settings. Our guide to AI crawlers and robots.txt covers that.

What is confirmed about adoption

This is where honesty matters, because a lot of marketing content presents llms.txt as essential for AI visibility.

What we can say with reasonable confidence as of this writing:

• No major AI search product (Google AI Overviews, ChatGPT search, Perplexity, Microsoft Copilot) has publicly documented that it reads llms.txt to decide what to cite or show.
• Google staff have said publicly that Google Search does not use llms.txt, and Google's documentation on AI features lists no special files or markup needed to appear in AI Overviews.
• Many websites, especially developer documentation sites, have published llms.txt files. Several documentation platforms generate them automatically.
• Some AI coding assistants and developer tools can use llms.txt or llms-full.txt when a user points them at a documentation site, because it gives them a clean summary to read.
• Some site owners report seeing requests for /llms.txt in their server logs. A request alone does not show how, or whether, the content is used.

What is unconfirmed:

• Whether any major AI engine uses llms.txt as a signal for citation or ranking
• Whether having one changes how often your brand appears in AI answers

AI platforms change their practices often. This could change, and it is worth checking each platform's documentation periodically.

Who might benefit today

Should you add one?

A practical way to decide:

Add one if:

• You run documentation or a technical product that developers use with AI tools
• Your site is already in good shape and this is a 30-minute task
• You want a tidy, curated summary of your site for any tool that does read it

Skip it, or put it at the bottom of the list, if:

• Your site has indexing issues, weak content or missing basics
• You expect it to get you cited by ChatGPT or Google
• Keeping it updated would become another stale file

There is little downside to having one, as long as it is accurate. The risk is opportunity cost: spending time on llms.txt instead of on things with clearer evidence behind them.

Common claims to be careful with

You will see these claims in vendor pitches and blog posts. Each one goes further than the evidence:

• "llms.txt is required to appear in AI search." No major AI search platform has documented this.
• "llms.txt works like robots.txt for AI." It does not control access at all.
• "Adding llms.txt increased our AI citations." Without a controlled test, other changes made at the same time could explain the result.
• "Google has adopted llms.txt." Google's Search documentation does not mention it as a signal.

If someone sells llms.txt as a core part of an AI visibility package, ask what evidence they have that it changed results.

What to do first instead

If your goal is AI search visibility, these steps have more support:

1. Make sure search and AI crawlers can access your site (robots.txt and firewall)
2. Get key pages indexed and ranking in Google and Bing
3. Rewrite important pages to answer questions directly
4. Keep your company description consistent across your site and profiles
5. Earn mentions on the third-party sources AI engines cite in your category

Our posts on what GEO is and how to audit your AI search visibility cover these in detail.

How to create an llms.txt file if you decide to

1. List the 10 to 30 pages that best explain your business: services, pricing, key guides, about and contact.
2. Write a one or two sentence summary of what your company does, for whom and where.
3. Create a Markdown file with your company name as the main heading, the summary as a blockquote, and grouped sections of links with one-line descriptions.
4. Upload it to the root of your domain so it loads at /llms.txt.
5. Check it opens as plain text in a browser.
6. Add a reminder to review it when you add or remove major pages.

Do not include private pages, internal tools or anything you would not want public. The file is visible to anyone.

If you want to know which AI search steps would actually move the needle for your company, book a business audit. We will tell you plainly whether llms.txt belongs on your list.