The consensus among experienced SEOs is that llms.txt doesn’t matter. SE Ranking, ALLMO, and OtterlyAI have all run studies and found no measurable impact on AI citations. John Mueller compared it to the keywords meta tag.
We wanted to validate that with our own data. But we also wanted to go further than prior studies, which largely stopped at “does llms.txt correlate with citations?” without controlling for prompt specificity, engine-level variation, or what actually does predict whether a site gets cited. We crawled 2,500 of the world’s most-visited websites, identified every confirmed llms.txt file, generated targeted prompts from each file’s content, and ran them through Perplexity, ChatGPT, and Claude.
What Is an llms.txt File?
An llms.txt file is a plain text file placed at the root of a website (e.g., yoursite.com/llms.txt) that gives AI systems a structured summary of what a site is about, what content it contains, and how it wants to be represented in AI-generated responses.
Think of it as a robots.txt for large language models (LLMs). Where robots.txt tells crawlers what to index, llms.txt tells AI what to understand. A site might use it to describe its core product areas, link to key documentation, or clarify what topics it’s authoritative on.
The standard was proposed in 2024 and gained traction as AI-powered search tools like Perplexity, ChatGPT, and Google’s AI Mode became significant traffic sources. The idea is that if AI engines can read a clean summary of your site, they’re more likely to cite you accurately. Whether that holds up in practice is what we tested.
How This Study Differs from Prior Research
SE Ranking, ALLMO, and OtterlyAI all asked variations of the same question: do sites with llms.txt get cited more? They found no measurable effect. But each study had limitations that left room for a follow-up.
SE Ranking’s study covered 300,000 domains but used a broad correlation approach without controlling for prompt specificity. If you’re checking whether Shopify gets cited when someone asks “What’s the best ecommerce platform?”, you’re testing brand authority, not whether llms.txt influenced the response.
ALLMO and OtterlyAI ran smaller, more targeted tests, but primarily against a single engine or a narrow set of queries.
Our study addresses three gaps:
- Prompt design tied to llms.txt content. We didn’t test whether llms.txt sites show up for generic queries. We fetched each site’s actual llms.txt file and generated 5 long-tail prompts based on the topics that file claimed authority over. If llms.txt influences citations, this is where you’d see it.
- Multi-engine testing. We ran every prompt through Perplexity, ChatGPT, and Claude independently. As the data shows, engine-level variation is massive and single-engine testing misses it entirely.
- Implicit control group. Every AI response cites multiple domains. By tracking all cited domains (not just our targets), we can compare citation rates for llms.txt adopters against the broader population of cited sites.
The Study at a Glance
| Metric | Value |
|---|---|
| Websites analyzed | 2,500 (top global sites by Ahrefs organic traffic) |
| Confirmed llms.txt files | 156 (6.5%) |
| AI-generated prompts tested | 656 |
| AI engines tested | 3 (Perplexity, ChatGPT, Claude) |
| Total prompt/citation checks | 2,128 |
| Citation rate for llms.txt domains | 34.3% |
6.5% of the Top Websites Have an llms.txt File
We crawled all 2,500 sites and confirmed 156 valid llms.txt files, a 6.5% adoption rate. This lines up with ProGEO.ai’s 7.4% across the Fortune 500 and SE Ranking’s 10.13% across 300,000 domains. SEO Strategy Ltd. puts total global implementation at 844,000+ sites growing 500%+ year-over-year, and both Yoast and Rank Math now offer one-click generation. Adoption is early.
SaaS and B2B companies are leading the way. Slack, Asana, Notion, Mailchimp, Hootsuite, Sprout Social, Shopify, and Dropbox all have files. It makes sense: these companies have product detail pages, feature documentation, and solution content that maps cleanly to the kind of structured summary llms.txt is designed for.
Social platforms are at zero. No Facebook, Instagram, TikTok, LinkedIn, Reddit, or X. That also makes sense. These are user-generated content platforms. There’s no single authoritative voice that could accurately represent what millions of users post every day in an llms.txt file, and AI engines already know what these platforms are.
82% of total organic web traffic comes from sites that haven’t adopted the standard at all.
llms.txt Sites Were Cited 34.3% of the Time
We fetched each site’s llms.txt, generated 5 long-tail prompts per domain, then ran all 656 through Perplexity (Sonar), ChatGPT (GPT-4o-mini), and Claude (Sonnet) via API. All calls used fresh sessions with no conversation history or account context. This matters because AI engines personalize citations based on prior queries and location, so a logged-in user who just asked about genealogy is more likely to see FamilySearch cited than a cold session would show. Fresh sessions give you a neutral baseline, not a prediction of what any specific user will see.
The prompts weren’t brand queries. If a site’s llms.txt covered pet GPS tracking, the prompt was “What’s the best GPS tracker for dogs with real-time location updates?” The goal was to put each domain in the best position to earn a citation.
| Engine | Citation Rate |
|---|---|
| Claude | 39.1% (323/826) |
| Perplexity | 32.3% (209/647) |
| ChatGPT | 30.2% (198/655) |
A third of the time, under ideal conditions, is not a strong signal for llms.txt driving citations.
AI Engines Disagree Dramatically
| Domain | Perplexity | Claude | Gap |
|---|---|---|---|
| bayut.com | 0% | 100% | Claude cites it every time; Perplexity never does |
| asana.com | 0% | 100% | Same pattern |
| cargurus.com | 80% | 0% | Perplexity cites it; Claude never does |
| shopify.com | 80% | 0% | Same pattern |
A domain can be cited 100% by one engine and 0% by another. Multi-engine optimization is a different discipline than traditional SEO, and you can’t test on one engine and assume the rest behave the same way.
This is the finding with the most immediate strategic implications. If you’re reporting on AEO performance using only one engine, you’re looking at a fraction of the picture. A site that appears to have strong AI visibility in Perplexity might be completely absent from Claude’s responses for the same queries.
92% of AI Citations Went to Sites Outside Our Confirmed llms.txt List
Every AI response cites multiple domains, so the non-target citations function as an implicit control group. Across 2,139 checks, AI engines cited 13,060 total domains. 92% went to sites outside our confirmed 156. The llms.txt domains captured 8%, versus a random expectation of 6.5%. That’s a 1.27x over-representation.
To put that in perspective: with 156 sites out of a population of thousands of potentially citable domains, a 1.27x lift doesn’t clear a meaningful statistical bar. The sample skews toward high-authority domains that are already more likely to be cited regardless of llms.txt. Without controlling for domain authority, content depth, and topical relevance, we can’t attribute that marginal lift to the file itself.
It’s worth being direct about what this means. If llms.txt were a strong signal, we would have seen a much larger gap between the two groups, especially since our prompts were specifically designed to match the topics each llms.txt file claimed authority on. This was the most favorable test possible. A 1.27x result under these conditions suggests the effect, if it exists at all, is minor.
Traffic Doesn’t Predict AI Citations
The correlation between a domain’s organic traffic and its AI citation rate was r=0.116. For context, a correlation of 1.0 means the two move in perfect lockstep. A correlation of 0 means no relationship at all. At 0.116, organic traffic has almost no predictive power over whether a domain gets cited by AI engines.
In practical terms: a site with 374K monthly organic visits outperformed a site with 15.9M. The sites earning consistent AI citations tend to have deep, well-structured informational content on specific topics, not the highest traffic numbers.
This has real implications for how you prioritize. Chasing traffic volume through broad keyword strategies won’t translate into AI visibility. What does translate is topical depth: being the most complete, most cited-by-other-sources authority on a specific set of questions.
What Actually Drives AI Citations
If llms.txt isn’t the lever, what is? Based on the patterns in our data and what we’ve seen in client work at Generix Marketing, three factors separate the domains that consistently earn AI citations from those that don’t.
None of them are new. As this piece on generative IR misinformation points out, good SEOs have always considered how search engines and NLP systems handle content preprocessing. Much of what’s being repackaged as “AEO tactics” has been part of solid SEO strategy for years for anyone who understands that ontology, structured text, and tabular data are how IR systems disambiguate content. The tactics that drive AI citations aren’t new. They’re the same fundamentals applied to a new set of retrieval systems.
1. Topical authority on questions AI needs to answer with a source.
AI engines don’t cite you because you exist. They cite you when they need to back up a factual claim and your content is the most credible source available for that specific claim. The sites with the highest citation rates in our study had deep, specific content on narrow topics, not broad content covering everything.
The practical move: identify the informational queries in your space where AI engines currently cite competitors. Build content specifically designed to be the best answer to those questions. Not a 500-word overview. The kind of content that other sites would link to as a reference.
2. Content structure that AI can parse and attribute.
AI engines are more likely to cite content when the answer to a question is clearly stated, well-organized, and attributable. Pages with clear headings, direct answers in the first few sentences of each section, and structured data (tables, lists, definitions) tend to outperform narrative-heavy pages.
This is different from traditional SEO content structure, where you might bury the answer below a long introduction to increase time on page. AI engines don’t care about time on page. They need to find, extract, and attribute a clear answer.
3. Track your Share of Model across every engine, not just one.
Share of Model (SoM) measures how often your brand appears in AI-generated responses compared to competitors for a given set of prompts. If you run 100 relevant queries and your brand shows up in 37 responses, your Share of Model is 37%. Think of it as the AI-era version of Share of Voice: instead of measuring ad impressions or search visibility, you’re measuring how often AI engines include you when answering questions in your category.
Our data shows why single-engine SoM is misleading. A domain cited 100% by Claude and 0% by Perplexity doesn’t have strong AI visibility. It has strong Claude visibility and zero Perplexity visibility. Those are different numbers with different implications, and rolling them into one score hides the gap.
You need to track Share of Model across every engine your audience uses: Perplexity, ChatGPT, Claude, and Google’s AI Mode. Each retrieves different sources, structures answers differently, and updates at different intervals. A single-engine SoM gives you a fraction of the picture.
Key Takeaways
- Only 6.5% of the top 2,500 websites have a confirmed llms.txt file. SaaS/B2B leads adoption. Social platforms: zero.
- We tested llms.txt under the best possible conditions (prompts derived from each site’s own llms.txt content) and saw a 34.3% citation rate. Even then, 92% of all citations went to sites outside the confirmed llms.txt list, with only a 1.27x over-representation that doesn’t clear a meaningful statistical threshold.
- Traffic doesn’t predict AI citations (r=0.116, effectively no correlation). A 374K-traffic site can outperform a 15.9M-traffic site when it has deeper topical authority.
- AI engines disagree dramatically on sources. A domain can be cited 100% by Claude and 0% by Perplexity for the same query. You need to test across all engines your audience uses.
- If your CMS has a one-click llms.txt toggle, turn it on. If not, spend that time building topical authority and testing your visibility across Perplexity, ChatGPT, Claude, and Google’s AI Mode individually.
- What drives AI citations isn’t a metadata file. It’s being the most credible, most structured source for the specific questions AI needs to answer with a citation. Build that, and track it across every engine.
