AI Visibility
Guide

llms.txt: what it is, how to write one, and what it won't do

AI Visibility team · November 10, 2026 · 4 min read

Short answer: llms.txt is a Markdown file you publish on your site (usually at /llms.txt) that gives AI agents a short summary of the site and a curated list of links to clean, readable pages. It was proposed by Jeremy Howard in September 2024. It is useful for AI tools that read your site on a user's request, such as coding assistants reading documentation. It is not a ranking signal: Google says you don't need AI text files to appear in AI Overviews or AI Mode.

What problem llms.txt tries to solve

Web pages are built for browsers: navigation, scripts, cookie banners, layout markup. An AI agent with a limited context window has to dig the useful text out of all that. The proposal's answer is a single file that says, in plain Markdown, what the site is and where the important content lives, ideally pointing at Markdown versions of the pages.

In the words of the proposal, the file is meant to "provide information to help agents use a website".

The format

The proposal (now in a second version, updated after two years of adoption) defines a simple structure, in this order:

  1. An H1 with the name of the site or project. This is the only required part.
  2. A blockquote with a short summary.
  3. Optional paragraphs with more detail (no headings).
  4. H2 sections with lists of links, each written as [name](url) with an optional note after a colon.
  5. By convention, an H2 called Optional for secondary links an agent can skip when it needs less context.

The file can sit at the site root or at a subpath: a file at /docs/llms.txt covers the pages under /docs/, and agents should use the most specific one.

The proposal also suggests serving a clean Markdown version of each page at the same URL with .md added (for example page.html.md), and linking those with rel="alternate" and type="text/markdown".

An example

Here is a small, realistic llms.txt for a software company:

# Acme Scheduling

> Online booking pages for clinics and salons. Plans from $12 per staff member per month. Integrates with Google Calendar and Outlook.

Acme is used by small service businesses to let clients book, reschedule and pay deposits online.

## Product
- [Features](https://acme.example/features.md): booking pages, reminders, deposits
- [Pricing](https://acme.example/pricing.md): plans, limits, what's included
- [Integrations](https://acme.example/integrations.md): calendars, payments, video

## Help
- [Getting started](https://acme.example/help/start.md): set up a booking page in 10 minutes
- [Cancellation policy settings](https://acme.example/help/cancellations.md)

## Optional
- [Changelog](https://acme.example/changelog.md)
- [Company](https://acme.example/about.md)

Notice what makes it useful: concrete facts in the summary (what, for whom, price), and links to pages that answer real questions.

Does llms.txt help you show up in ChatGPT or Google?

This is where most articles overpromise. Here is what the primary sources say:

  • Google: in its guidance on AI features, Google says there are no additional requirements to appear in AI Overviews or AI Mode, and that you don't need to create new machine-readable files, AI text files, markup or special schema.
  • OpenAI: visibility in ChatGPT search depends on OpenAI's search crawler, OAI-SearchBot, being allowed to crawl your pages. OpenAI's crawler documentation does not mention llms.txt.
  • The proposal itself does not claim search engine support. It reports adoption signals instead: thousands of sites publish the file, documentation platforms generate it automatically, and AI companies including OpenAI, Anthropic and Google publish llms.txt files for their developer docs. Its author expected the file to be mainly useful for inference (an agent reading your site to answer a question) rather than training.

So: publishing llms.txt is cheap and can help agents that read your site directly. It does not replace being crawlable, being useful, and being mentioned on the sites AI answers cite.

When llms.txt is worth doing

  • Developer documentation and APIs. Coding assistants are the clearest use case.
  • Sites with complex products or pricing, where a clean summary prevents an agent from quoting an outdated page.
  • Any site, as a low-cost extra. It takes an hour, and you control the summary of your own business.

How to write a good one

  1. Write the summary like a fact sheet: what you sell, to whom, the price range, the main limits. No slogans.
  2. Link the pages that answer buying questions: pricing, comparisons, integrations, policies.
  3. Prefer Markdown versions of those pages if you can generate them.
  4. Keep it current. A stale llms.txt is worse than none: an agent may trust it.
  5. Put secondary links under Optional.

Common mistakes

  • Treating it like robots.txt. It doesn't allow or block anything. Crawler access is still controlled by robots.txt and the crawlers' own rules.
  • Dumping every URL. It's a curated guide, not a sitemap.
  • Marketing copy in the summary. Agents need facts.
  • Expecting a traffic jump. Measure your AI visibility before and after if you want to know; don't assume.

What actually moves AI visibility

If your goal is to be named in ChatGPT and Google AI Overview answers, the work that shows up in the data is mostly off-site: the review sites, comparison articles, directories and threads that the answers cite. Our guide on how to get cited by ChatGPT covers that, and a free AI visibility audit shows which of those sources are cited for your competitors but not for you.

FAQ

Where do I put llms.txt? At https://yoursite.com/llms.txt, or at a subpath for a section of the site.

Will blocking AI crawlers in robots.txt conflict with llms.txt? They're independent. If you block OAI-SearchBot, OpenAI says your site won't be shown in ChatGPT search answers regardless of any llms.txt.

Sources