EnterraHost
llms.txt is a proposal, not a standard. It was published as a convention for telling a language model what a site is and where the useful parts are, and no standards body has adopted it. That matters for how much work it deserves. A file almost nobody reads is not worth a week, and a file that takes an afternoon is worth having before the crawlers that do read it arrive.
The convention lives at llmstxt.org. The short version is a Markdown file at the site root with an H1, a one line summary, and a set of links grouped under headings. There is no schema, no validator, and no required field beyond the title.
What it is not
It is not a replacement for robots.txt. robots.txt controls crawling and has a specification behind it, RFC 9309. llms.txt is a courtesy note. A crawler that ignores it is not doing anything wrong.
It is not a ranking factor and it does not make a page appear in an answer. It changes what a reader gets after it decides to fetch something, which is a smaller and more useful claim.
The shape that works
An index, not a dump. The reader is trying to decide where to look next, so every line should help that decision.
# Your Business
> Bookkeeping and payroll for small companies in the Western Cape.
We file monthly returns, run payroll for 1 to 50 employees, and answer
questions about SARS deadlines.
## Services
- [Monthly bookkeeping](https://yourbusiness.com/bookkeeping/): What is included, what it costs and what we need from you each month.
- [Payroll](https://yourbusiness.com/payroll/): Payslips, UIF and EMP201 submissions for teams of 1 to 50.
## Guides
- [What to send your accountant each month](https://yourbusiness.com/guides/monthly-pack/): The five documents that stop a return being late.
## Optional
- [Full site text](https://yourbusiness.com/llms-full.txt): every page above, concatenated.
The description after each link is the part most sites get wrong. It should say what the page answers, not repeat its title. A reader that has the title already does not need it twice.
What belongs in the file, and what does not
Pages that answer a question belong. Product pages belong if the product is what the site sells. Contact pages, privacy policies and terms belong in the file only if they answer something a reader would ask, which for most sites they do not.
Tag archives, paginated duplicates, cart and checkout pages do not belong. They are not destinations and they inflate the file. If a page carries noindex, it should not be in the index either.
There is a real argument for keeping the file short. Twenty entries that a reader might follow beat two hundred that it will not.
llms-full.txt, and why the cap exists
The companion convention is a second file holding the text of every page in one place. It is useful for a reader that wants the whole site without making thirty requests, and it is the file most likely to become unusable.
On a store with 2,000 products, the full text file is measured in megabytes. A reader with a context window will not finish it, and a large file is harder to keep correct than a small one. The plugin caps the file at 300 pages and says so at the end, which is a decision rather than a limitation of the format.
robots.txt and the Content-Signal line
WordPress can serve robots.txt from a filter, or a real file in the site root can win. When a file exists, the web server answers before WordPress runs and no plugin can add anything to the response.
That is why a setting can look applied while the live file says otherwise. The honest way to check is to fetch the file and read it, not to read the plugin's settings screen.
curl -s https://yourbusiness.com/robots.txt | tail -5
The Content-Signal line is a declaration about how the content may be used. It is a line in a text file, so it is a statement of intent rather than an access control. Crawlers that do not recognise it are required by RFC 9309 to ignore the line and keep parsing, which is the behavior you want from a line that is not part of the specification.
Building it on WordPress
The file is derived data. Everything in it already exists as a post, a page or a product, so the job is a query, a conversion and a cache, not a second content store.
Three details decide whether the output is right.
The source is post_content, not the rendered page. A WordPress page is mostly theme, and converting the finished HTML means every theme change changes the file. Reading the block tree gives the same Markdown on every theme.
The index and the sitemap should agree. Both are lists of pages the site wants read. Generating them from different queries is how a site ends up advertising a URL in one file and hiding it in the other.
Cache it, then invalidate on save. A rebuild on every request is wasteful, and a rebuild on a schedule is stale the moment somebody edits a page. Clear the cached file when a post is saved, and keep a lifetime as a safety net for changes that arrive by import.
The Enterra AI Visibility plugin does those three things and serves the result at /llms.txt, /llms-full.txt and a .md URL per page. It is free and it makes no external requests.
How to check the result
Fetch the file and read it, rather than trusting the screen that says it exists.
curl -sI https://yourbusiness.com/llms.txt
curl -s https://yourbusiness.com/llms.txt | head -40The content type should be text. The first line should be the site name. The links should be absolute and should resolve. If the file lists a page that returns a 404, the file is worse than absent, because a reader will spend its budget following the link.
Then edit a page and fetch the file again. A file that does not change after an edit is a file nobody is rebuilding, which is the failure mode that looks fine from a settings screen.
For the Markdown side, request one page with the header and compare it to the page.
curl -s -H "Accept: text/markdown" https://yourbusiness.com/about/ | head -20
What comes back should be the words, the title, the date and the canonical URL. If it is the full theme with a different content type, the negotiation is not working, and a cache is the usual reason.
When it is not worth doing
A site that is entirely behind a login does not need this. Neither does one that has no content anyone would ask a question about. The file is a description of what a reader can use, and a site with nothing to describe should skip it rather than publish an empty shell.
For everything else the cost is small and the downside is zero. The file is ignored by most crawlers today, and the ones that read it get a cheaper, more accurate view of the site. That is the whole trade.