SEO

How to use llms.txt for AI search optimization

What llms.txt actually is, the honest state of engine adoption, how to create one if you choose to, and where it fits in a real AI search strategy.

Lawrence Dauchy Lawrence Dauchy · · 10 min read
What llms.txt is and the honest state of AI engine adoption for AI search optimization

There is a lot of noise about llms.txt, the proposed file that supposedly tells AI engines what matters on your site. Some treat it as the new must-do for AI search; others call it a dud. The honest answer sits in between and is worth getting right before you spend time on it: llms.txt is a low-cost, optional experiment, not a confirmed ranking lever, and the biggest search player has said it does not use it. Here is what llms.txt actually is, how to use it if you choose to, and where it really sits in an AI search strategy.

The short answer

llms.txt is a proposed standard, a plain-text file at your domain root that offers AI systems a curated guide to your most important content. You can create one easily, but you should treat it as an optional, low-cost experiment rather than a proven optimization, because no major AI engine has publicly confirmed using it in production, and Google has explicitly said it does not. So if you want to add llms.txt as a cheap hedge and a tidy content summary, fine, but do not expect a ranking boost, and do not prioritize it over the fundamentals that genuinely drive AI visibility. The realistic stance is curious experimentation, not belief in a silver bullet.

What llms.txt is

Start with the proposal itself. As described at llmstxt.org, the standard is a markdown file placed at your site root that gives language models a concise, curated map of your key content and context, the idea being to help AI systems find and understand the most important pages without wading through your whole site. It is conceptually similar to how robots.txt guides crawlers or a sitemap lists pages, but aimed at LLMs and content comprehension rather than crawling rules. The proposal is genuinely elegant: a clean, machine-friendly summary of what your site is and what matters on it. The question is not whether the idea is reasonable, it is, but whether the engines actually use it.

The honest adoption status

This is where hype and reality diverge, so be precise. As Ahrefs documents, Google has said it does not support llms.txt and is not planning to, with its representatives comparing it to the long-discredited keywords meta tag. And as of now, no major AI provider has publicly committed to reading and acting on llms.txt in production, so widely repeated claims that specific engines rely on it are largely unverified. Adoption among sites is also modest, on the order of one in ten by some analyses. So the accurate picture is a reasonable proposal with low confirmed uptake by the engines that matter, not an established standard they all consume. Treat confident claims of big benefits with skepticism.

What Google says you actually need

The most authoritative signal cuts against treating llms.txt as necessary. Google’s own guidance on AI features is explicit that no special files are required: you do not need to create new machine readable files, AI text files, or markup to appear in these features, and there is no special schema.org structured data you need to add. An llms.txt file is exactly the kind of AI text file that statement covers. The real requirement, per Google, is being indexed and eligible to show with a snippet. So for Google’s AI surfaces at least, llms.txt is not a prerequisite, which reframes it from must-do to optional experiment. The official line is clear: do the fundamentals, not special files.

Claim versus reality

A quick reality check on the common claims.

Common claimReality
Major engines read llms.txtNot publicly confirmed; Google says it does not use it
It boosts your AI rankingsNo verified evidence of a ranking effect
It is required for AI searchNo; Google says no special AI files are needed
It is cheap and harmless to addLargely true, which is its main appeal

Read down the right column and the verdict writes itself: llms.txt is optional and unproven, but cheap, which is a reasonable case for a low-priority experiment, not a strategy.

How to create one, if you choose to

If you want to try it, it is simple. Create a markdown file named llms.txt at your domain root, and structure it as a concise guide: a short, accurate description of what your site or brand is, followed by curated links to your most important pages with brief descriptions, grouped logically. Keep it honest and current, mirroring your real, best content rather than stuffing it. Because it is plain markdown at a fixed path, it takes little time to make and maintain. That low cost is precisely why a careful experiment can be justified, you are not betting much, and you get a clean, human-readable summary of your site as a side benefit even if engines ignore the file.

Should you bother

Weigh it honestly against your priorities. The case for adding llms.txt is that it is cheap, low-risk, gives you a controlled summary of your content, and hedges against the possibility that engines adopt it more in future. The case against treating it as important is that the major engines have not confirmed using it, Google has said it does not, and there is no verified ranking benefit, so effort spent perfecting it is effort not spent on what works. The sensible position: if it costs you an hour, add a good one as a hedge; do not let it displace the fundamentals or believe vendor claims of guaranteed gains. It is a maybe-nice-to-have, not a lever.

What actually drives AI visibility instead

Put your real effort where the evidence points. The factors that genuinely drive AI visibility are the familiar ones: Ahrefs, across 75,000 brands, found AI visibility correlates closely with authority and relevance, and citations spread across well-structured content rather than any special file, with Ahrefs also finding only 38 percent of AI Overview citations come from top-10 pages. So being crawlable, genuinely authoritative, and answering real questions in clear, liftable content is what gets you cited, with or without llms.txt. This is the same conclusion as the broader markup question in structured data for LLM SEO: files and markup help at the margins; content and authority do the work.

Where llms.txt might still help

To be fair to the idea, there are mild upsides. It gives AI tools that do choose to read it, and your own internal AI workflows, a clean, curated view of your site, which can improve how they summarize you. It documents your key content and brand description in one place, useful beyond AI. And if adoption grows, early, well-made files cost nothing extra to already have. So llms.txt is not pointless; it is just unproven as a ranking lever. Treat its benefits as comprehension and hedging rather than direct AI-search gains, and you will value it correctly, neither dismissing it entirely nor overrating it.

Where it fits in a real AI search plan

To place llms.txt correctly, slot it at the bottom of a priority list, not the top. The high-impact work is being crawlable and indexed, answering the real question clusters in your niche, and building authority, the agenda laid out in why GEO is important. Researching the right questions to answer is the upstream step, where a free layer like the generative engine optimization tool helps, and structuring content for how AI assembles answers, the query fan-out effect, is what actually earns citations. Against that list, llms.txt is a five-minute optional hedge you do after the real work, not instead of it. Teams that invert this order, perfecting a file the engines may ignore while neglecting content, get the priorities exactly backwards, which is the most common llms.txt mistake.

A worked example

A team spent a week perfecting an elaborate llms.txt file expecting an AI visibility jump, and saw no measurable change, which the evidence would have predicted. Reassessing, they spent thirty minutes creating a clean, honest llms.txt as a low-cost hedge, then redirected the rest of their effort to the fundamentals: making pages crawlable, answering real questions in liftable passages, and building authority. Those changes moved their AI citations; the file did not. They kept the llms.txt because it was cheap and tidy, but stopped treating it as a strategy. The lesson was to size the effort to the evidence, a quick hedge for llms.txt, real investment in content and authority.

Common misconceptions

A few myths drive wasted effort. The first is that llms.txt is a confirmed ranking factor, when no major engine has publicly committed to using it and Google says it does not. The second is that it is required for AI search, when Google states no special AI files are needed. The third is that an elaborate file beats a simple one, when there is no evidence either moves rankings. The fourth is letting llms.txt displace the fundamentals that actually drive visibility. The fifth is trusting vendor claims of guaranteed gains. Clear these and you treat llms.txt as the cheap, optional experiment it is.

The bottom line

llms.txt is a reasonable, low-cost proposed standard, a curated markdown guide to your site for AI, but it is not a confirmed ranking lever, no major engine has publicly committed to using it, and Google has said it does not. So if you want to add a clean one as a cheap hedge and a tidy content summary, go ahead; just do not expect a boost or let it crowd out real work. Put your effort into being crawlable, authoritative, and clearly answering questions, which is what actually earns AI citations. Size your llms.txt effort to the evidence: a quick hedge, not a strategy, and revisit it only if the major engines actually announce they consume it.

Frequently asked questions

Do AI engines actually use llms.txt?

No major AI engine has publicly confirmed reading and acting on llms.txt in production, and Google has explicitly said it does not support it, with its representatives comparing it to the discredited keywords meta tag. Widely repeated claims that specific engines rely on it are largely unverified. So treat llms.txt as a proposed standard with low confirmed adoption by the engines that matter, rather than a file you can count on AI systems to consume and reward.

No. Google’s own guidance states you do not need to create special machine-readable or AI text files to appear in its AI features, and that the real requirement is being indexed and eligible to show with a snippet. An llms.txt file is exactly the kind of AI text file that statement covers. So it is not a prerequisite for AI visibility; the fundamentals of crawlable, authoritative, clearly-answered content are what actually matter.

Should I create an llms.txt file?

You can, as a low-cost hedge. It takes little time to make a clean, honest markdown file at your domain root summarizing your site and linking your key pages, and that gives you a tidy content summary plus a hedge if adoption grows. But do not expect a ranking benefit, since none is verified, and do not let it displace the fundamentals. The sensible approach is a quick, good file if you have an hour, with your real effort going into content and authority.

What should I prioritize over llms.txt for AI visibility?

Prioritize the proven drivers: make sure AI crawlers can access your indexable pages, write clear answer-first content that directly answers real questions, cover the question clusters in your niche, and build genuine authority and reputation. Studies show AI visibility correlates with authority and relevance, not special files, and citations come from well-structured content across the rankings. So invest there first; treat llms.txt as an optional, cheap experiment layered on top rather than a priority that competes with the fundamentals.

Sources

  1. The llms.txt proposal (llmstxt.org)
  2. AI features and your website (Google Search Central)
  3. What Is llms.txt, and Should You Care About It? (Ahrefs)
  4. Top Brand Visibility Factors in ChatGPT, AI Mode, and AI Overviews (Ahrefs, 75k brands)
  5. Update: 38% of AI Overview Citations Pull From The Top 10 (Ahrefs)

Frequently asked questions

Do AI engines actually use llms.txt?

No major AI engine has publicly confirmed reading and acting on llms.txt in production, and Google has explicitly said it does not support it, with its representatives comparing it to the discredited keywords meta tag. Widely repeated claims that specific engines rely on it are largely unverified. So treat llms.txt as a proposed standard with low confirmed adoption by the engines that matter, rather than a file you can count on AI systems to consume and reward.

Is llms.txt required to show up in AI search?

No. Google's own guidance states you do not need to create special machine-readable or AI text files to appear in its AI features, and that the real requirement is being indexed and eligible to show with a snippet. An llms.txt file is exactly the kind of AI text file that statement covers. So it is not a prerequisite for AI visibility; the fundamentals of crawlable, authoritative, clearly-answered content are what actually matter.

Should I create an llms.txt file?

You can, as a low-cost hedge. It takes little time to make a clean, honest markdown file at your domain root summarizing your site and linking your key pages, and that gives you a tidy content summary plus a hedge if adoption grows. But do not expect a ranking benefit, since none is verified, and do not let it displace the fundamentals. The sensible approach is a quick, good file if you have an hour, with your real effort going into content and authority.

What should I prioritize over llms.txt for AI visibility?

Prioritize the proven drivers: make sure AI crawlers can access your indexable pages, write clear answer-first content that directly answers real questions, cover the question clusters in your niche, and build genuine authority and reputation. Studies show AI visibility correlates with authority and relevance, not special files, and citations come from well-structured content across the rankings. So invest there first; treat llms.txt as an optional, cheap experiment layered on top rather than a priority that competes with the fundamentals.

Find the longtail searches your competitors ignore

Turn one seed keyword into hundreds of intent-grouped queries across SEO, AI Overviews, and GEO. Free forever for core research.

Generate free longtails