Skip to content
EnterraHost
Products
Tools
Hosting Email Support Client Area Find a domain

llms.txt for WordPress

llms.txt is a proposal, not a standard. It was published as a convention for telling a language model what a site is and where the useful parts are, and no standards body has adopted it. That matters for how much work it deserves. A file almost nobody reads is not worth a week, and a file that takes an afternoon is worth having before the crawlers that do read it arrive.

The convention lives at llmstxt.org. The short version is a Markdown file at the site root with an H1, a one line summary, and a set of links grouped under headings. There is no schema, no validator, and no required field beyond the title.

What it is not

It is not a replacement for robots.txt. robots.txt controls crawling and has a specification behind it, RFC 9309. llms.txt is a courtesy note. A crawler that ignores it is not doing anything wrong.

It is not a ranking factor and it does not make a page appear in an answer. It changes what a reader gets after it decides to fetch something, which is a smaller and more useful claim.

The shape that works

An index, not a dump. The reader is trying to decide where to look next, so every line should help that decision.

# Your Business

> Bookkeeping and payroll for small companies in the Western Cape.

We file monthly returns, run payroll for 1 to 50 employees, and answer
questions about SARS deadlines.

## Services

- [Monthly bookkeeping](https://yourbusiness.com/bookkeeping/): What is included, what it costs and what we need from you each month.
- [Payroll](https://yourbusiness.com/payroll/): Payslips, UIF and EMP201 submissions for teams of 1 to 50.

## Guides

- [What to send your accountant each month](https://yourbusiness.com/guides/monthly-pack/): The five documents that stop a return being late.

## Optional

- [Full site text](https://yourbusiness.com/llms-full.txt): every page above, concatenated.
The FindSignal dashboard inside the WordPress admin: readiness score, live llms.txt and Markdown URLs, and the template and robots state.
The dashboard inside the WordPress admin: readiness score, live URLs, the template and robots state, and the Markdown preview.
The full FindSignal dashboard in the WordPress admin, including agent activity and crawler access.
The same screen further down: agent activity and crawler access.
The FindSignal settings tab: included post types, the files, robots and the Markdown options.
The settings tab, where the included post types, the files and the robots block are set.

The description after each link is the part most sites get wrong. It should say what the page answers, not repeat its title. A reader that has the title already does not need it twice.

What the plugin adds

FindSignal serves the files from WordPress and reads the rendered page when the copy is not in the post, so a theme template, an Elementor page or a WPBakery page produces text instead of an empty body. The dashboard records which known agents requested the files and shows what robots.txt allows each of them. An editor can replace the generated Markdown for a single page without touching code, and the plugin serves ai.txt as a pointer for agents that ask for it. The settings screen saves switches and selects immediately, with a confirmation toast.

What belongs in the file, and what does not

Pages that answer a question belong. Product pages belong if the product is what the site sells. Contact pages, privacy policies and terms belong in the file only if they answer something a reader would ask, which for most sites they do not.

Tag archives, paginated duplicates, cart and checkout pages do not belong. They are not destinations and they inflate the file. If a page carries noindex, it should not be in the index either.

There is a real argument for keeping the file short. Twenty entries that a reader might follow beat two hundred that it will not.

llms-full.txt, and why the cap exists

The companion convention is a second file holding the text of every page in one place. It is useful for a reader that wants the whole site without making thirty requests, and it is the file most likely to become unusable.

On a store with 2,000 products, the full text file is measured in megabytes. A reader with a context window will not finish it, and a large file is harder to keep correct than a small one. The plugin caps the file at 300 pages and says so at the end, which is a decision rather than a limitation of the format.

robots.txt and the Content-Signal line

WordPress can serve robots.txt from a filter, or a real file in the site root can win. When a file exists, the web server answers before WordPress runs and no plugin can add anything to the response.

That is why a setting can look applied while the live file says otherwise. The honest way to check is to fetch the file and read it, not to read the plugin's settings screen.

curl -s https://yourbusiness.com/robots.txt | tail -5

The Content-Signal line is a declaration about how the content may be used. It is a line in a text file, so it is a statement of intent rather than an access control. Crawlers that do not recognise it are required by RFC 9309 to ignore the line and keep parsing, which is the behavior you want from a line that is not part of the specification.

Building it on WordPress

The file is derived data. Everything in it already exists as a post, a page or a product, so the job is a query, a conversion and a cache, not a second content store.

Three details decide whether the output is right.

Read the post's own content first, and the rendered page only when the copy is not in it. A normal page converts from its blocks, so a theme change does not change the file. A template or builder page has no copy in the post, so the rendered page is the only honest source. Converting every page's finished HTML would make the output follow the theme.

The index and the sitemap should agree. Both are lists of pages the site wants read. Generating them from different queries is how a site ends up advertising a URL in one file and hiding it in the other.

Cache it, then invalidate on save. A rebuild on every request is wasteful, and a rebuild on a schedule is stale the moment somebody edits a page. Clear the cached file when a post is saved, and keep a lifetime as a safety net for changes that arrive by import.

The FindSignal plugin does those three things and serves the result at /llms.txt, /llms-full.txt and a .md URL per page. It is free and it makes no external requests.

How to check the result

Fetch the file and read it, rather than trusting the screen that says it exists.

curl -sI https://yourbusiness.com/llms.txt
curl -s https://yourbusiness.com/llms.txt | head -40

The content type should be text. The first line should be the site name. The links should be absolute and should resolve. If the file lists a page that returns a 404, the file is worse than absent, because a reader will spend its budget following the link.

Then edit a page and fetch the file again. A file that does not change after an edit is a file nobody is rebuilding, which is the failure mode that looks fine from a settings screen.

For the Markdown side, request one page with the header and compare it to the page.

curl -s -H "Accept: text/markdown" https://yourbusiness.com/about/ | head -20

What comes back should be the words, the title, the date and the canonical URL. If it is the full theme with a different content type, the negotiation is not working, and a cache is the usual reason.

When it is not worth doing

A site that is entirely behind a login does not need this. Neither does one that has no content anyone would ask a question about. The file is a description of what a reader can use, and a site with nothing to describe should skip it rather than publish an empty shell.

For everything else the cost is small and the downside is zero. The file is ignored by most crawlers today, and the ones that read it get a cheaper, more accurate view of the site. That is the whole trade. If you would rather not hand-write the file, the free WordPress plugin serves it from your existing pages, and it is listed with the rest of what we publish on the plugins page.

This page is about WordPress. The same file exists for Joomla, and the Joomla guide covers what differs there. The files are written into the site root rather than served through a route, and Joomla has no sitemap of its own.

Summarize this guide with

You need an account with the assistant you choose. It fetches the guide itself, and you send the prompt, not us.

Every guide in this category is on the FindSignal page.

Something here wrong, or out of date? Tell us at editorial@enterrahost.com.