What matters
- Google says llms.txt is not required for its AI search experiences.
- The file does not control access, indexing, or main URLs.
- Keep it small, correct, and tied to your main site links.
The useful version
The llms.txt proposal describes a Markdown file that can list a site’s most important public pages. It can name the site, explain it, and link to guides, data, tools, policies, and contact details. The proposal is public, but broad adoption and ranking effects have not been proven.
It is easy to keep current when the site builds it from the same source as its links. It may help tools that choose to read it.
The claims it cannot support
The file does not prove that a bot read the pages. It does not prove that a model trained on them, a search tool indexed them, or an AI will cite them. Google says its AI search does not need a special AI file. It gives llms.txt no special role.
It is also not a security tool. Keep private pages behind a login and safe server rules.
A sane implementation
- Keep the file in version control or build it the same way each time.
- Link only canonical public resources.
- Use ordinary Markdown headings and links.
- Include a short content/license note when relevant.
- Test every link in continuous integration.
- Update it when navigation and priority resources change.
- Measure requests and later results before doing more.
This site has an llms.txt file because it costs little to maintain. The pages it links to still do the real work.
Use the technical AEO audit to check the page, sitemap, robots rules, initial HTML, and live edge before spending more time on this optional file.
Sources
- The llms.txt proposal, llms.txt
- Top ways to ensure your content performs well in Google's AI experiences, Google Search Central