What is a llms.txt?

At Webtify, we get that question more and more often. That’s why we just explain it, what it is, why the big AI companies aren’t doing much official work with it yet, what might change in the future and how we handle it ourselves.

What is a llms.txt?

A llms.txt is a small text file that you place in the root of your website. Similar to a robots.txt, which tells search engines like Google which pages they may or may not “visit. Only llms.txt is not meant for Google, but for AI models such as ChatGPT, Claude and Gemini.

The idea behind it is actually quite simple. Websites are cluttered. Navigation menus, footers, cookie banners, pop-ups, it’s all in between. When an AI model visits your Web site to gather information, it has to cut through all that noise. A llms.txt gives the system a neat, structured summary of who you are, what you do and which pages are most relevant. Sort of like a sitemap but for the future.

Why are large AI companies still not doing much with it?

This is where it gets interesting, and also a little crazy maybe because it exists but is not used. More than 800,000 thousand websites have now created a llms.txt. Well-known names like Cloudflare and Stripe use it. Anthropic, the company behind Claude, even has it on their own documentation website. Yet, no major AI model (company) has officially confirmed that their systems actively use this txt file when responding to searches.

Crawlers from OpenAI and Microsoft do pick up the file. But whether they really do anything with it, or whether it is just standard crawl behavior, no one knows for sure. A top Google executive has even indicated that no AI model uses llms.txt at this time.

What might change in the future?

That it will be used is certain, only when, that is still a very good question.

People are searching less and less through Google and increasingly asking their questions directly to ChatGPT, Gemini or Perplexity. If your company does not appear in those answers, you are missing a growing portion of your potential customers. That’s why Webtify has included a GEO in its SEO subscription as a standard so we can help our clients properly.

llms.txt may in the future become the way you as a company determine what AI will tell about you. Google has already included it in their Agent2Agent protocol, the framework for AI agents using the Web independently. That does signal that the txt is being taken seriously, even if its official use is not yet known.

How Webtify is handling it now

We currently see no compelling reason to actively deploy llms.txt for our customers. Not because we think it is nonsensical, but because the practical added value has not yet been proven. There is no data to show that a llms.txt is already producing more customers or better AI visibility.

What does work, and what we do focus on, is having good FAQs on your website. With our Website Business subscription, we often make sure that a FAQ page is already developed so that it works well for different AI models.

AI models such as ChatGPT, Gemini and Claude pick up their answers from content that is clear and structured. An FAQ with the right questions and answers is exactly the content that AI models understand and absorb.

So if you want your company to be found better in AI searches, don’t start with a technical file on your server. Start by putting yourself in the customer’s shoes, what are they going to ask, and make sure those questions are answered clearly on your website.

As soon as llms.txt is seriously embraced by the (major) players, we will, of course, immediately join in.