If you sell online, someone has probably told you to add an llms.txt file so ChatGPT and friends will recommend your products. It is a plain text file at the root of your site that lists your important pages for AI tools. It costs nothing to add. The question is whether anything reads it.
Ahrefs looked. In May 2026 they checked the server logs of 137,210 domains that use their analytics and bot tools, and recorded every request for /llms.txt.
What Ahrefs found
About 28% of those domains (38,360) had a valid llms.txt. Of those files, 97% received zero requests during the month. Only around 1,100 domains saw any fetch at all.
And the few requests that did arrive mostly were not from AI assistants. 96% came from bots, but most of those were SEO auditors, tech profilers and general crawlers. AI bots of every kind made up 19.5% of the requests. The bots that fetch pages when a shopper asks an assistant a question, the ones you actually want, barely showed up:
Ahrefs' explanation is simple: AI tools fetch llms.txt when a link, an index or a user tells them it exists. They do not go looking for it. Coding agents read them because developers point them at documentation; shopping questions do not work that way.
Two honest caveats from the study itself. Ahrefs customers are more technical than the average site, so the 28% adoption figure is probably high. And the study counted requests, not whether a bot used the content. Still, a file nobody requests cannot help you. If you already have one, leave it; it does no harm. Just do not count it as your AI plan.
What to check instead (about 20 minutes)
For an AI assistant to mention your product with the right price, three things have to be true: its bot is allowed in, the bot is not stopped at your CDN or host, and the page it gets back actually contains the product facts.
1. Your robots.txt lets the search bots in (5 minutes)
Open yourshop.com/robots.txt in a browser. Look for blocks that name AI bots. OpenAI, for example, uses separate bots for separate jobs:
- OAI-SearchBot finds pages to show in ChatGPT's search results. If you block it, you are asking not to appear there.
- GPTBot collects content that may be used to train models. Blocking it is a separate decision, and many shops make it.
- ChatGPT-User visits a page when a person asks ChatGPT about it. OpenAI says robots.txt rules may not apply to it, because a user started the visit.
So you can say no to training and yes to being found. A robots.txt that does that looks like this:
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /On Shopify, the default robots.txt already keeps bots out of cart, checkout and account pages and lets them reach products and collections. You only need to edit it (through the robots.txt.liquid theme template) if you or an app added AI blocks. On WooCommerce, check both the file and any SEO plugin that writes it.
2. Your CDN is not silently blocking them (5 minutes)
Since 1 July 2025, new domains on Cloudflare block known AI crawlers by default. If your shop's domain went through Cloudflare after that date, your robots.txt may say welcome while the edge says no. Log in, find the AI crawler setting (AI Crawl Control) and look at what it is set to and which bots it has been stopping. Other hosts and security plugins have similar bot switches; check yours.
Blocking is a real choice, not a mistake to fix by reflex. If you would rather not have your catalogue used by AI companies, keep it. Just know that you are also stepping out of AI shopping answers.
3. Price, stock and specs are in the HTML (10 minutes)
This is the step that is easy to miss. Vercel's analysis of AI crawler traffic found that OpenAI's crawlers and Anthropic's ClaudeBot fetch pages but do not run JavaScript. Google's Gemini and Applebot do. If your price, variant list or in stock label only appears after a script runs (some page builders, review widgets and B2B pricing apps work like this), those bots see an empty slot.
Check one real product page with a command that, like those bots, does not run scripts:
curl -s -A "Mozilla/5.0" https://yourshop.com/products/your-product | grep -o -i -E '"price"[^,]{0,40}|in stock|out of stock' | headIf you see your price and stock status, those bots can see them too. If you see nothing, open the page with JavaScript turned off in your browser and look at what is missing. Most standard Shopify and WooCommerce themes put price and the product description in the HTML; app-injected content is the usual gap.
While you are on the page, read it as an assistant would. Does the text say what the product is made of, its sizes, who it is for, how long delivery takes, and the return terms? An assistant can only repeat facts that are written down.
What we still don't know
Nobody outside the AI companies knows exactly how they pick which shop to mention, and it changes often. What the Ahrefs data does show is that a file most AI tools never fetch is not where the effort should go. Letting the right bots in and making sure each product page states its facts in plain HTML are the steps you can check yourself.
