llms.txt: Does It Actually Work for GEO? (Evidence-Based, 2026 Update)

Bottom line: llms.txt does not improve AI search visibility. Google has said so on three separate occasions, a 300,000-domain correlation study found no measurable link to citations, and five independent server-log studies — including 1,227 logged requests across roughly 900 domains in August 2026 with zero verified frontier-lab crawlers — recorded no real retrieval use. Publish it for coding agents if you wish; do not expect citations from it.

llms.txt does not work for GEO. Google's May 2026 AI Search Optimization Guide (John Mueller) explicitly confirmed that llms.txt carries no special weight for AI citation selection. A SERanking study of 300,000 domains found no measurable correlation between llms.txt presence and AI citation rates. A 48-day server log study (wislr.com, Feb–March 2026) recorded 12,099 AI bot requests and zero requests to /llms.txt. The accumulating evidence is unequivocal: llms.txt does not improve AI search visibility.

The scale of what you are missing is now measurable. Google AI Overviews serve an estimated ~13 billion impressions every month globally (Nico Digital, 2026), and AI search across the major engines processes 3.5B+ queries every week (Axis Intelligence, June 2026). Within that discovery surface, third-party publishers earn 6.5× more AI citations than owned domains (Axis Intelligence, 2026) — earned media, not a self-described summary file, is what wins visibility. A file you write about yourself is the weakest possible citation signal.

This guide walks through what llms.txt was supposed to do, what every major study says in 2026, Google's official position (multiple times on record), and where to invest your optimization effort instead. The BlueJar three-layer technical GEO stack framework (Feb 2026) provides a useful lens: llms.txt sits in the "context layer" — the layer with the least proven citation impact — while robots.txt (crawl layer) and content quality (entity layer) carry the real weight.

September 2026 Update: Five Log Studies, One Null Result — And Adoption Data That Explains The Confusion

One reason the llms.txt debate never settles is that adoption numbers contradict each other without anyone being wrong. The rate depends entirely on which cohort you sample. Digital Applied measured a fixed 219-host panel on August 3, 2026 and found 51.8% adoption (113 of 218 reachable hosts) — but that panel is deliberately developer-skewed. The same tracker put adoption on the Tranco top 1,000 at 8.7% (87 of 1,000) as of June 2026, or 15.8% counting only the 549 reachable roots. Two other large samples sit in between: Ahrefs found 28% of 137,210 domains publish a valid file, and SE Ranking put it at 10.13% across roughly 300,000 domains. Same file, four denominators, one conclusion: adoption skews toward large brands and developer documentation, not the average site.

The consumption side of the handshake is where the evidence is uniformly negative. Across roughly 900 monitored domains, 1,227 requests were logged to llms.txt-family files — and zero came from a verified frontier-lab crawler. The largest single requester was a commercial data aggregator at 794 requests; the second was ordinary human browser traffic from people checking the file existed. Ahrefs’ 137,210-domain panel found the same shape: 96% of the requests that did arrive came from non-AI-retrieval bots, breaking down as SEO audit tools 21%, unidentified bots 14%, traditional crawlers such as Googlebot 13% and tech-profiling tools such as BuiltWith 11%, with AI retrieval bots at roughly 1%. The sharpest illustration: Slackbot fetched llms.txt more often than PerplexityBot did.

The cleanest single-property test remains Semrush’s. It added an llms.txt file to its own property, Search Engine Land, in March 2025 and read the server logs from mid-August to late October 2025. Google-Extended, GPTBot, PerplexityBot and ClaudeBot all recorded zero requests; only Googlebot and Bingbot visited, a handful of times, with no sign of special treatment. Google’s position has also hardened: John Mueller called llms.txt "purely speculative for now" in June 2026, said it is "not done for search," and repeated the keywords meta tag comparison — while Google’s 2026 site-owner guidance on AI features now names llms.txt directly in a mythbusting section.

"No major LLM provider currently supports llms.txt. Not OpenAI. Not Anthropic. Not Google."

What survives is the use case the spec was written for. Chrome ships a Lighthouse audit for llms.txt under a new agentic-browsing category, Google’s Agent2Agent protocol references the format, and Mintlify, GitBook, Wix and Yoast all ship generators. Anthropic and OpenAI publish their own files for their developer docs — proving they are comfortable as authors, not that their crawlers go looking for one on your domain. If you run API documentation, ship the file for coding agents. If you are chasing citations in ChatGPT or AI Overviews, five log studies say it will do nothing.

Sources: Digital Applied, "llms.txt in Practice: Adoption Data, Evidence, and Setup" (measured August 3, 2026). Ahrefs, 137,210-domain llms.txt study (June 2026). SE Ranking, ~300,000-domain adoption and correlation study. Semrush, Search Engine Land llms.txt server-log test (March to October 2025). John Mueller, Google (June 2026 and May 2026). Google Search Central guidance for site owners on AI features (2026).

What is llms.txt

llms.txt is a proposed Markdown file at /llms.txt that describes your site to language models, proposed by Jeremy Howard in September 2024. It was designed as a context-saving map for coding agents — never as a ranking or citation signal, which is why the evidence against it as a GEO tactic is so consistent.

llms.txt is a proposed standard for a Markdown-formatted file hosted at /llms.txt on the root of a domain. The intent was to give AI models a curated, machine-readable summary of a site's content — the way robots.txt tells crawlers what they may access, llms.txt would tell models what the site is about.

The proposal gained traction in mid-2024 among SEO practitioners searching for a GEO equivalent of robots.txt. Several major sites published llms.txt files, and tooling emerged to generate them automatically. The hypothesis: AI engines would fetch llms.txt and use it to improve retrieval accuracy. By 2026, the evidence against this hypothesis is decisive.

Google's official position (multiple statements)

Google has rejected llms.txt three times across two years: Gary Illyes (July 2025) said Google does not support it and is not planning to, John Mueller (June 2025) compared it to the keywords meta tag, and the May 2026 AI Search Guide called it a content description helper, not a citation signal.

Google has stated its position on llms.txt multiple times across two years:

Source: Google Search Central — John Mueller AI Search Guide (May 2026). Gary Illyes at Search Central Live Deep Dive (July 2025). John Mueller social post (June 2025).

"llms.txt is a content description helper, not a citation signal. AI does not give it special weight. The best way to get cited by AI is to write content that provides genuine information gain."

The SERanking study: 300,000 domains

SERanking compared AI citation rates across 300,000 domains and found no measurable correlation with llms.txt presence — removing the feature actually improved their XGBoost model, meaning it added noise. Adoption was 10.13% and flat across traffic tiers (9.88% small sites vs 8.27% at 100K+ visits).

SERanking ran the largest empirical test of llms.txt effectiveness in 2025. They analyzed 300,000 domains, comparing AI citation rates between sites that published llms.txt and sites that did not, controlling for content quality, domain authority, and topic.

"We found no measurable correlation between llms.txt presence and AI citation rates across the 300,000 domains studied. The signal was indistinguishable from noise. In fact, removing llms.txt from our XGBoost predictive model improved accuracy — meaning it added noise, not signal."

Only 10.13% of domains had an llms.txt file. Adoption was flat across traffic tiers — 9.88% for small sites vs. 8.27% for sites with 100K+ monthly visits. High-authority sites actually adopted it less. The finding is consistent with how AI search engines actually work: models like ChatGPT Search, Perplexity, and Google AI Overviews retrieve and cite from their indexed corpus, not from a site-provided summary file.

The wislr.com server log study (2026)

Server logs settle the "do they fetch it" question directly. Over 48 days (February to March 2026), wislr.com recorded 12,099 AI bot requests and exactly zero requests to /llms.txt. Robots.txt was fetched thousands of times over the same window.

The most recent empirical evidence comes from a 48-day server log study (February–March 2026) by wislr.com. They recorded 12,099 AI bot requests from multiple crawlers across their server logs and analyzed which files those crawlers requested.

Source: wislr.com server log study (Feb–March 2026), cited in BlueJar technical GEO analysis.

Caveat: Some site operators (e.g., SEO Ray Martinez) report OpenAI polling their /llms.txt file every ~15 minutes, suggesting that crawling behavior varies by site and crawler. However, even when /llms.txt is fetched, no study has demonstrated a citation lift from its presence. The wislr.com data shows that for this site, across 48 days and 12,099 bot requests, no AI crawler attempted to fetch the file.

Which AI engines support llms.txt

No major AI search engine supports llms.txt as a retrieval or citation signal. Google has said no three times; OpenAI, Perplexity and Microsoft have made no announcement; Anthropic has published its own file but never stated that Claude reads external ones.

Sources: Google Search Central (May 2026, June 2025). Gary Illyes at Search Central Live (July 2025). OpenAI, Perplexity, Anthropic documentation.

The Ahrefs June 2026 log study: 97% never read

Ahrefs analysed 137,210 domains and found 97% of published llms.txt files received zero requests in May 2026. Of the rare requests that did arrive, 96% came from bots and AI retrieval bots accounted for just 1.1% — the file is being written far faster than it is read.

The most recent and largest crawler-log evidence comes from Ahrefs (Ryan Law, June 15, 2026), which analyzed the server logs and live traffic of 137,000 domains. The headline finding: 97% of published llms.txt files received zero requests in May 2026 — no bots, no humans, nothing. Of the 3% that were read, 96% of the requests came from bots rather than people, and AI retrieval bots — the category most associated with AI search citations — accounted for just 1.1% of those rare requests. The single largest AI consumer was agentic infrastructure such as Claude Code, pointing to a developer-tooling use case rather than a citation one.

Real-world proof: citation logic lives in retrieval weighting, not files

Citation logic sits upstream in retrieval weighting. After the GPT-5.6 rollout on August 8, 2026, ChatGPT’s site: fan-out jumped from ~0.3–0.5% to 16–17% in a day and Reddit’s citation share fell 86% in a week — decisions no self-published hint file can influence.

A separate, real-world signal from August 2026 makes the "retrieval over files" point concrete. Promptwatch's live-UI citation monitoring found that after the GPT-5.6 rollout on August 8, 2026, ChatGPT Search's use of the site: operator in its query fan-out jumped from roughly 0.3–0.5% to 16–17% of all fan-out queries in a single day — a ~46× increase (Promptwatch, Aug 2026). When an engine scopes a search to a named domain, the decision about which site to ask is made upstream in retrieval weighting, not by anything the target site publishes in a hint file. The same shift coincided with Reddit's ChatGPT citation share collapsing from a 3.83% baseline to 0.52% (an 86% relative drop) within a week — aggregated community content lost ground precisely because the model began targeting specific authoritative domains directly rather than retrieving broadly. This is the inverse of what a file-based hint would do, and it is exactly why the three independent log studies above all point the same way.

Source: Promptwatch citation tracking, "Why Did ChatGPT Stop Citing Reddit?" (Aug 2026). Third-party measurement of an undocumented change; treat the 16–17% figure as directional, not an OpenAI-published metric.

Where llms.txt actually works: agentic browsing, not search

The one real consumer is agentic tooling, not search. Chrome’s Lighthouse 13.3 added an experimental Agentic Browsing audit in May 2026, and Mintlify, GitBook, Wix and Yoast auto-generate the file. Coding assistants fetch it when a developer points them at it — manually, not automatically.

There is exactly one environment where llms.txt demonstrably gets used — and it is the environment it was designed for: AI developer tools and coding agents. Chrome's Lighthouse 13.3 (updated May 2026) added an experimental "Agentic Browsing" audit that checks for llms.txt, describing it as "an emerging convention." But this is about automated agents navigating your site to complete a task, not about search ranking or AI Overview citations. Lighthouse only flags a server error for the file; a missing file is marked "Not Applicable," not failed.

In practice, AI coding assistants — Cursor, Windsurf, Claude Code, GitHub Copilot — fetch llms.txt when pointed at documentation sites, and LangChain ships an MCP server built around exposing these files to agents. Anthropic recommends the format in its guidance on writing for agents. The honest framing: llms.txt is agent-readiness infrastructure, not search-visibility infrastructure. If your site is developer documentation or an API reference, ship one; if you are chasing ChatGPT or Google AI Overview citations, the evidence says it does nothing.

The three-layer technical GEO stack

Technical GEO has three non-interchangeable layers: the crawl layer (robots.txt, highest impact), the entity layer (Schema.org, medium impact) and the context layer (llms.txt, lowest proven impact). Fix them in that order — a hint file cannot compensate for a blocked crawler.

BlueJar's 2026 analysis provides a useful framework for understanding where llms.txt fits — and does not fit — in a GEO strategy. Technical GEO has three non-interchangeable layers:

1. 1.
   Crawl layer (robots.txt) — Access control. Gates whether AI engines can fetch your pages at all. Highest impact. Block a retrieval bot and you are invisible regardless of content quality.
2. 2.
   Entity layer (Schema / structured data) — Labels who you are and what a page contains. Helps indexing and knowledge graph building. Medium impact. Does not drive direct citations but aids discovery.
3. 3.
   Context layer (llms.txt) — A curated summary of your site for language models. Lowest proven impact. No study has demonstrated measurable citation lift from its presence.

The BlueJar analysis concluded: "Get the crawl layer right first. Then invest in content quality and structured data. llms.txt is an optional signal with no proven citation effect — treat it accordingly."

Why llms.txt fails the GEO test

Three structural reasons: AI engines cite indexed content rather than self-authored summaries, self-description is trivially gameable (Illyes’ keywords meta tag comparison), and no engine has announced that it consumes the file as a retrieval signal.

Three structural reasons explain why llms.txt cannot deliver the visibility lift its proponents claimed:

1. 1.
   AI engines cite from indexed content, not summaries

   Citation requires verifiable text the re-ranker can attribute. A self-authored summary at /llms.txt is not citable — AI engines would be quoting the site about itself. Google's May 2026 guide confirmed this: citation selection is based on indexed content quality, not self-described summaries.

2. 2.
   Self-description is trivially gameable

   A site could claim anything in its llms.txt. AI engines cannot trust self-description as a ranking signal. They rely on independent retrieval from indexed content. Gary Illyes compared it to the keywords meta tag — a signal that was abandoned precisely because it was gameable.

3. 3.
   No engine has announced consumption as a citation signal

   Google (×3 statements), OpenAI, Perplexity, and Microsoft have not announced that they consume llms.txt as a retrieval signal. Even Anthropic, which published its own llms.txt, has not stated that Claude reads external llms.txt files for citation selection.

Where to invest instead (2026 update)

Invest in tactics with measured lift: original statistics (+156% for AI Overviews per Authoritas), expert quotations (+41%), statistics (+33%), structured FAQ (+44%) and fluency (+29%). Google’s 2026 Information Gain framework rewards original data, case studies and unique insight.

The Princeton GEO study quantified the lift from strategies that actually work. Google's May 2026 guide adds one more: Information Gain — content that provides original value, data, or insights not already available on the web. Google I/O 2026 reinforced this at the agent level: agent-readable pages need clean semantic HTML — a single H1, logical H2 hierarchy, and keyboard-navigable forms — not a machine-readable summary file.

The window for cheap tricks is closed. AI referral traffic grew 357% year-over-year in 2025 (theworlddata.com), and AI Overviews now appear on up to 48% of queries (Digital Applied, March 2026). The click cost is now measurable too: AI Overviews cut position-1 organic click-through rate by 58% as of December 2025 (Ahrefs, Feb 2026) — brands that get cited win the remaining clicks; those that don't get none. With that much discovery happening inside AI answers, the only durable lever is content that earns citation on its own merits — not a self-described summary file that no engine reads.

Sources: Authoritas — AI Overview citation-factor study (2026). Aggarwal et al., KDD 2024 (Princeton GEO-bench). BrightEdge — AI Overview citation rates (2026). Google AI Search Guide (May 2026). SERanking 2025 llms.txt study. wislr.com server log study (Feb–March 2026). BlueJar technical GEO stack analysis (Feb 2026).

If you still want to publish llms.txt

llms.txt is harmless but unproven, so publish it only after every higher-impact strategy is in place. Keep it accurate and aligned with your indexed content — never use it to make claims your visible pages do not support.

llms.txt is not harmful — it simply does not help measurably. If you have implemented every higher-impact strategy and have spare time, publishing an llms.txt costs minimal effort. The risk is opportunity cost: time spent on llms.txt is time not spent on strategies with proven lift, or — more importantly in 2026 — on creating content with genuine Information Gain.

If you publish one, keep it accurate, concise, and aligned with your visible site content. Do not use llms.txt to make claims that do not appear in your indexed content — at best it is ignored, at worst it could be flagged as inconsistent if AI engines ever do consume it. Google's May 2026 guide categorizes it as a "content description helper" — treat it as such.

GeoAura's own testing reached the same conclusion: the durable GEO levers are indexed, well-structured, citation-worthy content — not self-described summary files. For the platform-agnostic playbook, see our AI Search Optimization guide, and for the strategies with measured lift, the 9 GEO optimization strategies breakdown quantifies each one.

Frequently asked questions