llms.txt and AI-readable pages

An llms.txt that tells AI assistants what your site is

An llms.txt file is a plain-text map at the root of your domain, written for language models rather than people. This work drafts that map from your real pages, then goes further: it turns pages whose content only appears after JavaScript runs into clean markdown, and it checks that the machine-readable copies agree with the live site.

It's done in Claude Code, with plain Node scripts for the parts that have to be exact. None of it asks a model to guess what your pricing is.

Why an llms.txt file isn't enough on its own

Support for the llms.txt file across AI assistants is uneven and poorly documented, so the file is treated as one layer rather than the plan. Its bigger value is the discipline of writing it: deciding, in about a hundred lines, what your company is and which pages prove it.

The layers underneath matter as much. A pricing page built from sliders and dropdowns can look empty to anything that doesn't run the page's scripts. And a site that blocks crawlers gets nothing from a perfect map, which is exactly what drafting one for a client turned up.

How a draft gets written

Each draft follows the same order, and starts by distrusting the sitemap:

1. Gather the sitemap, the homepage headline copied word for word, the product pages, the API page, pricing and the about page.
2. Record whether an API, SDKs or an MCP server actually exist, rather than implying them.
3. Structure it by what a visitor is trying to do: products, developers, solutions, pricing, comparisons, resources and company.
4. Trim the sprawl: link a glossary of 100 or more entries once, cut a blog of 50 or more posts to the 10 to 15 that matter, and point to an index instead of listing 30 sub-pages.
5. Give every link a one-line description, and end with notes telling AI systems what not to assume.

The target is 100 to 150 lines, modelled on a well-known public file that runs to 143. Drafts so far run from 81 to 186 lines across five sites, and each ends with a note of what the client must verify before it goes live. One is confirmed live at its site's root so far.

What the gathering step catches

Writing the map is also an audit. On one site the self-serve product page, arguably the most important page for a buyer, was missing from sitemap.xml, so anything reading the sitemap alone would never have found it.

On another, the site returned a 403 to crawlers. An llms.txt file there would have been a map to a locked building, so the fix belonged to the server before it belonged to the file.

Pricing pages as markdown

Pricing is where AI assistants most often get a company wrong, because the numbers sit inside interactive components. For Reply.io, a script reads the pricing page's content blocks through the WordPress API, read-only, and renders every slider stop, dropdown option and billing period into markdown with a quick-reference table.

Then it checks itself. The script fetches the live page again and compares every plan title, tier label, price and account count against its own output, and fails on any mismatch. Any block type it doesn't recognize is marked as unrendered rather than skipped. The main pricing file runs to 1,005 lines.

Machine-readable documentation for AI agents

Some companies now publish documentation written specifically for AI agents: every page with a markdown twin, plus llms.txt and a full-text version. For Reply's agent documentation site, the work was writing the build specification it adopted, auditing the result, and shipping fixes.

One fix was structural. Every canonical tag, sitemap entry and structured-data URL pointed at a redirect, so Search Console reported the entire sitemap as redirects. The fix pointed all of them at the final address, added links that open any page's markdown in five AI assistants, and generated FAQ schema from the pages' own FAQ sections, covering 273 questions across 61 pages.

The checks that keep it honest

Machine-readable copies drift from the real site quietly, so each layer carries a check:

A build that fails any of these doesn't ship, which is the only reliable way to stop a model from learning last quarter's prices.

When AI assistants get you wrong

Do this when AI assistants already describe your product and get the details wrong, especially pricing, plans or what integrates with what. It is also a good first step for any company with an API, since developers increasingly ask an assistant before reading the docs.

It won't fix a site that blocks crawlers or a product nobody mentions yet. For the second, start by measuring where you appear with the AI visibility tracker.

What a first llms.txt costs

A first llms.txt file takes a day or two, most of it reading the site. The pricing and documentation checks are a larger build, measured in days per page type rather than weeks.

The work sits in execution capacity, the AI agents route at a flat $2,000 a month, added to a fractional Head of Content retainer.

Start with how an assistant describes you now

Ask two AI assistants what your product costs and what it integrates with, and bring the answers to a call. We'll compare them with your live pages and decide which layer to fix first.