Guide / llms.txt
Does llms.txt help AI search?
Does llms.txt help AI search? Not on its own, in our data. 46% of cited Houston business sites had an llms.txt file, against 33% of sites that were not cited, but the difference disappeared once content factors such as reviews, prices and city pages were taken into account. Google says Search ignores the file.
An llms.txt file is a Markdown file at the root of a site that points AI agents to its most useful pages. We tested whether having one goes with AI citations on 880 Houston business websites. Here is what we found, why, and what to do instead.
02
Data guide
Proof before claims
01 / What it is
What an llms.txt file is.
It is a plain-text summary of a website, written for language models rather than people.
An llms.txt file is served at /llms.txt. It gives a short description of the site and a list of links to the pages that matter, so an AI agent can find the right page without reading the whole site. Some sites also publish llms-full.txt, the full text of those pages in one file.
The format was proposed by Jeremy Howard of Answer.AI on September 3, 2024, at llmstxt.org. It is a proposal, not a web standard. Its clearest use is developer documentation, where coding assistants fetch it to get API details right.
Google’s guide to optimizing for generative AI features says you don’t need llms.txt or other special files to appear in Google Search, including its AI features, because Google Search doesn’t use them. It adds that keeping one for other services will neither help nor harm your visibility in Google Search.
02 / What we tested
How we tested it on 880 Houston sites.
As part of the Houston AI Citation Study, we checked 40 commonly recommended factors on business homepages on October 30, 2026, the day the questions were asked, including whether each site published an llms.txt file.
- 01
Cited sites: 492 Houston-area business websites that Google AI Mode or Perplexity cited for 232 local questions.
- 02
Comparison sites: 388 Houston businesses in the same 12 industries that were not cited, taken from the census behind the Houston AI Visibility Index.
- 03
Tests: the share of each group with each factor, odds ratios adjusted for industry and link authority, and whether each factor still mattered once the others were known.
03 / Result
What the data shows.
On its own, llms.txt went with citations, with an odds ratio of 1.9. Once the content factors were in the model, the effect disappeared. The factors below did hold up.
| Measure | Cited sites | Not cited | Held up with other factors known? |
|---|---|---|---|
| llms.txt published | 46% | 33% | No |
| Reviews or testimonials on the homepage | 75% | 46% | Yes (odds ratio 3.4) |
| Prices shown | 35% | 18% | Yes (odds ratio 2.5) |
| Years in business stated | 53% | 36% | Yes (odds ratio 1.8) |
| Meta description | 96% | 79% | Yes |
| City pages in the sitemap | 69% | 46% | Yes |
| AI search crawlers allowed in robots.txt | 98% | 92% | No; too few sites block them to measure |
The whole technical group (robots.txt, firewall, speed, HTTPS and llms.txt) added nothing to the prediction once content was known. Source: AI citation factor test, 880 Houston-area business homepages in 12 industries, odds ratios adjusted for industry and link authority. Data: doi.org/10.5281/zenodo.23820175 (CC BY 4.0).
04 / Why
Why llms.txt looked like it mattered.
- 01
Well-kept sites do many things at once. Owners who add an llms.txt file also tend to keep reviews, prices and city pages current. The file travels with the work that matters.
- 02
Plugins add it automatically. Some SEO plugins now create the file without the owner asking, so it may mostly mark sites that use those plugins. We did not measure that directly.
- 03
Local answers come from elsewhere. Google AI Mode’s picks came from Business Profiles, and ChatGPT and Claude leaned on directories. A file that summarizes your site does not change those sources. See which websites AI assistants cite.
05 / What to do
What to do instead, and when llms.txt is worth it.
- 01
Do the six basics first
Reviews on the homepage, a specific business type in your schema, prices, years in business, city pages and a meta description held up in every test.
Tip: Our answer engine optimization guide covers each one with the numbers.
- 02
Keep AI search crawlers allowed
Almost every cited site allowed them in robots.txt. Blocking them by accident, often through a firewall or bot-protection setting, stops them reading your pages.
Tip: Check robots.txt and your firewall rules for GPTBot, OAI-SearchBot, PerplexityBot and ClaudeBot.
- 03
Publish llms.txt if agents use your site
If developers or AI agents use your documentation, product catalog or booking flow, a small llms.txt that links the key pages can help them. It costs little, and Google says it neither helps nor harms Search.
Tip: One line about the business, then links to your service, city, pricing and contact pages, each with a few words.
- 04
Measure with blind questions
Ask the questions your customers ask, without your business name, on each engine, more than once, and watch which pages get cited.
Tip: Change one thing at a time so you can tell what made the difference.
We publish an llms.txt file on this site ourselves. It is there for agents, not because it moves citations.
07 / Questions
Common questions.
Short answers you can act on.
Does llms.txt help AI search visibility?
Not on its own, in our data. 46% of cited Houston business sites had one, against 33% of sites that were not cited, but the difference disappeared once content factors such as reviews, prices and city pages were taken into account.
Does Google use llms.txt?
No. Google’s guide to its generative AI features says Google Search doesn’t use llms.txt or similar files, and that keeping one for other services will neither help nor harm your visibility in Google Search.
What is the difference between llms.txt and robots.txt?
robots.txt tells crawlers what they may fetch. llms.txt is a voluntary summary of your most useful pages for AI agents. Blocking AI search crawlers in robots.txt stops them reading your pages; adding llms.txt did not get sites cited.
Should a small business create an llms.txt file?
Only after the basics. It is quick to make and does no harm, but in our data it did not raise citations once reviews, prices, city pages and meta descriptions were in place.
What should an llms.txt file include?
A title line with your business name, a short summary, and lists of links to your key pages, each with a few words on what it covers. Keep it accurate and update it when pages change.
Can llms.txt hurt my site?
Not in Google Search, which ignores it. The main risk is an out-of-date file that points agents to pages that no longer exist.
Next / Review
Fix what moves citations.
We’ll check your site against the factors that held up and tell you which ones to fix first.