AI Search

How to write an llms.txt file that AI crawlers can use

4 September 2026 · 1 min read ·

An llms.txt file is a plain text document at the root of your site that tells AI crawlers what your company does and which pages answer what. It is a small file with an unusually high ratio of effort to effect.

Why it exists

An assistant reaching your site has to work out, from HTML built for humans, what you sell and which page explains it. Navigation, cookie banners and marketing copy get in the way. llms.txt is a direct answer to that problem: a short, structured description written for machines.

What goes in it

Start with an H1 of your company name, then one paragraph describing what you do in plain language. Follow with your key pages, each as a link with a one-line description of what a reader will find there. Keep the main file small – under two kilobytes is a reasonable target.

The longer companion

A second file, llms-full.txt, carries the detail: full product descriptions, differentiators, frequently asked questions, who you serve. There is more room here, but the same rule applies – write facts, not adjectives. An assistant summarising your positioning statement produces nothing a buyer can use.

Where to put it

Both files belong at the root: /llms.txt and /llms-full.txt. Serve them as text/plain. Then check they are actually reachable – a file that returns your 404 page as HTML is worse than no file, because it looks present to a casual check.

Keeping it current

An llms.txt describing last year’s product is a liability. Regenerate it when your pages or positioning change, and verify after each deploy that what is live is what you meant to publish.