llms.txt for your site: why it will not make AI cite you — and why it still matters
In a year, llms.txt collected more myths than almost any other SEO file. Some sell it as a magic button: drop it in the site root and ChatGPT or Perplexity will start citing your brand. Others run experiments and conclude that nobody reads it.
The truth sits in the middle. The file does something — just not what miracle-sellers promise. Here is what it is, who actually fetches it, and how much it moves AI mentions.
What llms.txt is
llms.txt is a Markdown file at yoursite.com/llms.txt. It is a short cheat-sheet for language models: who you are, what you do, which pages matter, and how to interpret your content.
| File | What it does |
|---|---|
| robots.txt | Rules for crawlers: what they may and may not index |
| sitemap.xml | A list of pages so search engines miss nothing |
| llms.txt | A meaning map: who you are and which pages matter. A model reads it only if it fetches the file — it does not replace crawling HTML |
So llms.txt answers three questions at once: what this site is, what matters most, and what is safe to cite. Jeremy Howard proposed the format in September 2024. It is still a standardization proposal, not a universally adopted standard.
llms.txt vs llms-full.txt
The spec has two files, and people mix them up.
| File | What it is |
|---|---|
| llms.txt | A compact map. A snapshot of the site: name, a two-line summary, links to key sections. Aim for under 30 KB |
| llms-full.txt | A full dossier. Important content inlined as Markdown, not just links. That can run to hundreds of kilobytes |
The map says “read these pages.” The dossier says “read it here, skip crawling.” A full file pays off for docs, manuals and article hubs. For a service site or a shop it is usually wasted work — there is no information core worth packing into one file.
Who actually reads llms.txt
Most AI crawlers barely touch it. That shows up in logs, not in sales copy.
An Ahrefs study of server logs across 137,000 domains in May 2026 found that about 97% of published llms.txt files got zero requests. Of the requests that did land, AI-search bots barely feature. Coding agents, ordinary crawlers and SEO tools checking whether the file exists show up more often. GPTBot, ClaudeBot and Googlebot still crawl HTML and links.
The file is not useless — the useful cases are narrow:
Docs and knowledge bases.
When an agent looks up an API or a product detail, a ready map saves crawling.
RAG and AI agents.
A system that assembles its own context will take a markdown map over raw HTML. That is where the “insurance” is real: lower odds of the wrong page or a stale line. ChatGPT answering from memory usually never reads the file.
SaaS product docs
same story.
On shops, landings and local service sites the file will not manufacture a brand mention. Crawlers already reach them the usual way.
What affects citations
To stop expecting miracles from llms.txt, it helps to see where citability actually comes from. This is not a Google ranking and not a measurement. It is an expert order of investment: what moves the chance of being cited, and what only tidies a decision already made. More on source selection: how AI engines select sources.
- Brand and off-site mentions — decide whether you enter the source pool. Without this, models almost never pick you.
- Content depth and structure — what the model builds an answer from: complete pages, facts, question–answer logic.
- Schema.org / JSON-LD — how a machine reads the page: organization, article, prices, FAQ.
- Page freshness — in fast-moving topics, current pages land in answers more often than archives.
- Load speed — affects crawling, barely the decision to cite you.
- llms.txt — a map for RAG and AI agents. It does not force a citation; it lowers the odds of the wrong page.
If llms.txt were a “get into AI answers” button, it would sit first. It sits last: a map for agents and RAG, not a reach lever. The insurance works when a tool already went to your site and fetched the file — not when ChatGPT answers from memory.
When to create the file — and when to skip it
Create it if you have
- documentation, knowledge bases, guides, product FAQs;
- a SaaS, API or technical product that agents query;
- a content site whose core pieces you want models to see in compressed form.
Do not expect an effect if you have
- a local service site — clinic, law firm, repairs;
- an online store with hundreds of SKUs;
- a landing or promo page.
For those types the file will not pull visibility and will not replace a brand. It is not harmful — it is just the wrong job. It is also code: a root file, not a CMS edit. What an owner does alone vs hands off: promoting your site in AI search: what to do yourself vs hand off. For a broader plan, see how to get into AI answers.
How to create and check llms.txt
If you decide to add it, the process takes about 10 minutes.
File structure
In Markdown:
- first line — an H1 with the site name;
- right below — a blockquote: who you are, what you do, which topics you own;
- then sections with headings and links to key pages.
What to write
Facts only: products, audience, which URLs to cite first. Skip “we are a team of professionals who put quality first.” The model will take the file literally — give it facts, not ads.
A simple llms.txt example:
# CloudSync API
> HTTP API docs for a file-sync product aimed at developers.
> Core sections: authentication, endpoints, rate limits and request examples.
## Docs
- [Quickstart](https://example.com/docs/quickstart)
- [Authentication](https://example.com/docs/auth)
- [API reference](https://example.com/docs/reference)
## Product
- [Pricing](https://example.com/pricing)
- [Status](https://example.com/status)How to publish
Put llms.txt in the site root so it opens at yoursite.com/llms.txt. It must be reachable by a direct link, with no login wall and no extra redirects.
How to check
Open yoursite.com/llms.txt in a browser. Confirm:
- the response is 200, not 403/404;
- the markup is valid: H1, a short summary, sections with links;
- robots.txt does not block the file.
Check your site’s llms.txt online for free
Enter a domain — we request GET /llms.txt and check whether the file exists and follows the format.
If you need the full picture of how AI sees your site, run a free GEO / AEO audit. llms.txt is only a small part of the available tools.
How SEOFetcher’s audit treats llms.txt
In the report, llms.txt is a hygiene signal: the file exists, the markup is valid, the URL is directly reachable. It sits next to structure, schema.org and E-E-A-T.
It does not move AI visibility. Brand, content and markup still lead. It answers a different question: is a map available for agents if they come looking.
See how readable your site is for AI
A GEO / AEO audit shows which readiness points you already cover — including llms.txt, JSON-LD and E-E-A-T.
Short answers to common llms.txt questions are below.
Frequently asked questions
Does ChatGPT read this file?
Most of the time, no — not during a standard crawl. RAG systems, AI agents and tools that specifically look for the file do use it. Treat it as a citation guarantee and you will be disappointed.
Is llms.txt required?
No. It is a proposal, not a mandatory standard. A site without llms.txt is still indexed and can still appear in AI answers.
Will llms.txt replace sitemap or robots.txt?
No. Sitemap lists pages for search engines, robots.txt controls crawler access, llms.txt explains meaning for AI. They work together, not instead of each other.
Will llms.txt replace schema.org / JSON-LD?
No. Schema.org JSON-LD lives on the pages and explains how to read the content — organization, article, facts. llms.txt sits at the site root and points to what matters. They are read at different times, in different ways. The file cannot replace markup or repair missing schema. For the basics, see the plain-language guide to schema.org.
Can adding the file hurt anything?
No practical harm has been documented. The file is tiny and does not interfere with normal indexing. The real mistake is blocking it in robots.txt — then a check will correctly show it as unavailable.
Should I add it if the effect is weak?
Usually yes for docs and SaaS: a cheap map for agents in about 10 minutes. For a service brochure you can skip it — it will not move citations. The file does not grow reach and does not replace schema.org.
Can llms.txt be part of a GEO strategy?
Yes, as a last stroke — not the strategy. Brand mentions, structure, schema.org and fresh content do the heavy lifting. llms.txt helps if a RAG system or agent fetches the file. A practical list is in the GEO optimization checklist.