What the file is
llms.txt is a text file in the root of a site, opened at yoursite.com/llms.txt. Inside is a short account of what the company does and a list of important pages: a link and a one-line note for each. The thinking behind it is that a model finds it easier to start from such a map than to dig through a whole site with its layout and scripts.
The format is deliberately plain. A title, a quoted description, then sections of links. It is ordinary text a person can read too.
What it is not
One caveat gets lost often: it is not a standard. No consortium adopted it and nobody is obliged to read it. It is a convention, supported by some and ignored by others.
The practical consequence is that we do not know which model, which crawler, or when, opens the file, and we cannot promise that having one makes a site appear more often in assistant answers. Anyone promising that is promising something they do not know.
How ours is built
Ours is not written by hand. Code generates it from the same data that renders the pages: services, case studies, articles, market pages and a block of facts about the agency. Publish an article and it appears in the file without a separate edit, so the file cannot drift away from the site.
We also put a block of facts at the top: who we are, where we work, which languages we build sites in, which awards the agency has. The logic is that when an assistant is asked which agency to pick, it finds it easier to name one it can say something verifiable about. A list of pages alone does not give it that. Whether a model uses the block this way, we do not know, but writing the facts down in one place costs us nothing.
Next to the file sit markdown mirrors. Every article and service page is also available at the same address ending in .md — the same text with no layout. That is not a map but the material itself, in its cleanest form.
What actually helps
Set the file aside and a machine cares about more important things.
Finished HTML from the server. If a page is assembled by a script in the browser, a crawler may see an empty shell. No llms.txt fixes that: it points at a page that still cannot be read.
Structure. Proper headings, lists, tables. The machine can tell where an answer starts and ends.
Direct answers. A question-and-answer block is easier to quote than a meaning spread over paragraphs. More on this in how we make websites visible to ChatGPT and AI search.
Markup. Structured data says the same as the copy, without guesswork. How it works is in structured data explained.
Open access. Some sites block AI crawlers in robots.txt by accident, together with ordinary scrapers. Check that your rules do not forbid more than you meant.
How not to hurt yourself with it
Three mistakes come up most. First: describing things the site no longer has, such as a service that was withdrawn. Second: linking pages that are blocked from indexing or return an error. Third: writing slogans instead of description. A model needs names, things and addresses, not "best-in-class solutions for your business".
A good file reads like an honest reference: what we do, for whom, where to read more.
Do you need one
A short answer: build it, but not first. The order:
- Make sure pages arrive as finished HTML. Easy to test: open a page with scripts disabled and see whether the text is still there.
- Fix structure and give direct answers to what clients ask.
- Add markup.
- Then llms.txt, if someone can keep it current.
That last point matters more than it looks. A stale file with links to deleted pages is worse than none. If nobody can maintain it by hand, generate it from the site's data, as we do.
We build this layer on our own sites and can build it on yours: how we build websites.
