llms.txt Checker, Generator & Monitor: AI Search Readiness

Audit and generate llms.txt — the emerging standard that tells AI answer engines what your site is and where to find its key pages. Check whether a site exposes a good llms.txt / llms-full.txt, score its AI-search readiness 0–100 with concrete fixes, and draft a new llms.txt straight from its own sitemap.

What it does

llms.txt is the robots.txt-style file AI assistants read to understand a site and find its most important pages — a title, a one-line summary, and sectioned markdown links. Most sites do not have one, and most that do never had it validated. This server checks whether a site's llms.txt (and its optional llms-full.txt companion) is actually well-formed per the llmstxt.org spec, scores it, and can draft one from scratch — all without ever crawling the site page by page.

Who it's for

Marketing and SEO teams. Audit your domain, then generate the file the audit says is missing, so assistants describe your product from pages you chose.

Documentation teams. Generate from your docs sitemap so assistants answering user questions land on current pages rather than stale ones.

AI-visibility auditors and agencies. Score a client or prospect's site and hand them a concrete punch list before an engagement.

Developers and AI agents. Run audit_llms_txt in CI after a robots.txt change and fail the build if the score drops, so an AI-crawler block never ships unnoticed.

When to use it, and when not to

Use it to score a site's AI-search readiness, to draft a publishable llms.txt from a site's own pages, and to check many sites at once.

Do not use it as a ranking tool or a guarantee of citation. A high score means assistants can read the site, not that they will cite it. Generated files are drafts for review — sectioning comes from URL structure and summaries from the site's own metadata.

Why it's built this way

• Spec-accurate scoring, not a guess — validates the actual llms.txt structure (H1 title, > summary blockquote, ## Section groups of markdown links) and scores readiness 0–100 against that spec. The scoring is a considered view, published openly in each result rather than presented as an industry standard.
• Honest about "missing" vs. "couldn't tell" — a 404 on /llms.txt (NOT_FOUND) is never confused with a DNS or timeout failure (UNREACHABLE) or an anti-bot wall (BLOCKED). Most naive checkers collapse all three into one generic failure.
• Draft, not crawl — generate_llms_txt builds a draft from the site's own /sitemap.xml (capped at 200 URLs) plus its homepage title and description. It never crawls individual pages, so it is fast, memory-light, and polite to the target site.
• Nothing stored — every fetch is capped, streamed, and discarded after the response.
• Bulk-ready — audit up to 50 sites in one call, 5 at a time, with a pass/fail summary.

Tools

audit_llms_txt — fetch a site's /llms.txt and /llms-full.txt, validate structure against the spec, and score readiness 0–100 with concrete issues and fixes. Also notes /sitemap.xml and /robots.txt presence for context.

{ "url": "example.com" }

{
  "status": "OK",
  "hasLlmsTxt": true,
  "hasLlmsFullTxt": false,
  "readinessScore": 70,
  "issues": [
    {
      "severity": "medium",
      "message": "Missing a '> summary' blockquote under the title.",
      "fix": "Add a one-line '> summary' blockquote describing the site, right after the title."
    }
  ]
}

status is one of OK, NOT_FOUND, UNREACHABLE, BLOCKED, or INVALID_INPUT.

generate_llms_txt — build a DRAFT llms.txt from a site's own /sitemap.xml (capped at 200 URLs) plus its homepage title and meta description. No deep crawl.

{ "url": "example.com" }

Returns { input, status, generatedLlmsTxt, pagesUsed, notes }.

bulk_audit_llms_txt — audit up to 50 sites in one call, 5 at a time, with a pass/fail summary.

{ "urls": ["example.com", "another-site.com"] }

Use cases

• AI-search readiness audits — score a client or prospect's site and hand them a punch list.
• Pre-launch / pre-redesign check — confirm llms.txt still resolves and scores well after a migration.
• Drafting a first llms.txt — get a real starting point built from the site's own sitemap and homepage copy.
• Portfolio monitoring — audit a list of client sites, a competitor set, or a company's own subdomains.

FAQ

What is llms.txt? A plain-text file at /llms.txt that gives AI assistants a structured map of a site — a title, a one-line summary, and links grouped under ## Section headings — so they can find and cite the right pages instead of guessing from a full crawl.

Does it crawl my whole site? No. generate_llms_txt reads only /sitemap.xml (or a Sitemap: line in robots.txt) and the homepage — never individual pages — by design.

What's llms-full.txt? An optional companion file with the full text of key pages. The audit checks for it and flags it as a low-priority improvement when missing.

Do I need an API key? No — this checks public files only and needs no credentials.

Trust & limits

Reads public files only — llms.txt, llms-full.txt, sitemap.xml, robots.txt, and the homepage — never anything behind a login or paywall. Nothing is stored: every fetch is capped and streamed, and page content is discarded once the response is built. This is an AI-readiness aid, not an SEO or legal guarantee.