llms.txt is a proposed format for a short, machine-readable overview of website content, with explanations and links for AI agents. It may help locate documentation and important pages. Its existence alone does not establish better visibility, citations or brand recommendations.
My criterion is simple: before creating it, I want to know which tool it helps, for what task and how we will assess benefit. If the answers are vague, I would first invest in a clear offer, accessible pages and reliable company information.
What does an llms.txt file contain?
The llms.txt proposal describes a Markdown document with the site name, an optional short summary and organised links to detail. It may sit at the site root or a relevant subpath. The proposal also covers links to readable Markdown versions.
Imagine product documentation with clear getting-started, API and limitations sections. A short map may help an agent using it find the right section. That is a specific, testable task. The claim that the same file will automatically make a company a preferred choice in every AI answer is much weaker.
For a business website, I would select pages describing the actual offer, team, contact and important rules. Generic praise such as “the best agency” does not clarify a service. Important information must also be accurate on the original pages.
llms.txt, robots.txt and sitemaps have different roles
| File | Practical role | What it does not prove |
|---|---|---|
| robots.txt | Communicates access rules to automated visitors that respect it | That content is high quality or recommended |
| sitemap.xml | Helps search engines discover listed URLs | That every URL will be indexed |
| llms.txt | Provides a curated content overview to agents using it | That a brand will be cited or recommended |
I would retain this distinction in agency proposals. Setting up three files produces three technical deliverables. A business finding needs further checks. For sitemaps, I use Google's explanation; for crawler access, the robots.txt documentation.
Check that linked pages exist and summaries match them. If a service, terms or URL changes, the supporting map must change too. An outdated map adds another source of incorrect information.
Text description of the graphic
- First check page access, indexing, content and clear company information.
- Then test llms.txt for a specific tool.
- A supporting content map. No citation guarantee.
What does Lighthouse actually check?
In the Lighthouse llms.txt documentation, this check concerns AI agents browsing websites. A server error fetching the file is flagged, while a missing file returning 404 is marked not applicable because the file is currently optional.
Read a tool's recommendation in context. I would not infer from a Lighthouse check that llms.txt is required for organic placement or a Google AI citation.
Google Search Central separately states that appearing in AI search features does not require a new AI text file. Helping an agent use a website and qualifying for search appearances are different questions.
When would I recommend creating llms.txt?
When there is a clear use case. A company may have extensive technical documentation used with AI tools, and a selected tool may support this input. We can then compare finding accurate information with and without the file.
For a small service-business website, I would first check core company information. If the services, suitability and contracting process are unclear, summarising the same content will not solve the main problem.
A small experiment may also make sense after foundations are sound. It should answer a specific question at a reasonable maintenance cost. I would not present it as a substitute for GEO strategy and AI visibility checks.
How would I test usefulness?
- Define the task. For example, find accurate integration terms or a particular service page.
- Record the tool and conditions. Version, internet access, input links and instructions affect the result.
- Save a baseline. Record answer accuracy, sources and missing information.
- Repeat with the file. Where possible, verify that the tool actually fetched or used it.
- Compare multiple attempts. One improved answer may be variation; conclusions must reflect test size.
This is a proposed test, not an experiment already performed. A URL request in a server log confirms retrieval, not influence on an answer. An accurate answer alone also cannot establish where information came from.
For public brand visibility, I would use separate GEO measurement. One test assesses a map's usefulness for a particular tool; the other examines how a company appears in relevant answers.
What do I ask an agency proposing llms.txt?
I ask for a priority rationale, file content, maintenance plan and definition of a successful test. If bundled with other technical changes, distinguish verified fixes from experiments.
In an AI visibility audit, I would record it as a specific recommendation with a reason and verification method. Clients can then implement it now, later or reserve the budget for a more important obstacle.
Frequently asked questions about llms.txt
Does llms.txt guarantee a ChatGPT recommendation?
A file's existence does not justify promising a recommendation. Check documented support and actual use by the specific tool, and assess recommendations within a documented question sample.
Should I remove an existing file?
First check accuracy, accessibility and maintenance cost. A correct supporting document need not be removed merely because its citation impact is unproven. Assign it the appropriate role.
Should the whole website be copied into one file?
Not automatically. Start with a concise map and the needs of the tool using it. More text is not inherently more useful, particularly when it contains duplicates or outdated information.