llms.txt is a Markdown text file that organizes site information for AI search engines, placed at your site's root. Note that Google has officially stated llms.txt is not required for its generative AI search, and no other AI vendor (OpenAI, Anthropic, etc.) has issued formal support as of May 2026. That said, ChatGPT crawler access to llms.txt has been observed on our own sites — making it a low-cost forward-looking measure rather than a proven requirement.
What Is llms.txt?
llms.txt is a Markdown file designed to help AI systems understand your site. Placed at your site's root (/llms.txt), it provides a summary of your site, links to key pages, and document structure for AI consumption. The specification was published by llmstxt.org in 2024 and is rapidly gaining adoption.
robots.txt
Controls crawler access. Tells bots which pages to crawl or skip.
llms.txt
Provides site information to AI. Tells AI agents what your site is about and where to find key content.
Where llms.txt Stands Today
As AI search adoption grows, the industry is debating how best to communicate with AI agents. llms.txt is one such option, with the specification published by llmstxt.org in 2024. However, as of May 2026, no AI search vendor has officially declared support for llms.txt, and its effectiveness has not been quantitatively demonstrated.
Current Status (May 2026)
- ・Google: Officially stated llms.txt is not required for its generative AI search (AI Overviews/AI Mode) (Official guide)
- ・OpenAI / Anthropic / Perplexity: No official support statements
- ・Observed: ChatGPT crawler access to llms.txt has been confirmed on our own sites (citation impact unproven)
- ・Our position: Low-cost forward-looking measure worth deploying, despite unproven effectiveness
See our lab report "Does llms.txt Improve GEO? — 300K Domain Analysis" for detailed evidence.
For our analysis of the "llms.txt is unnecessary" debate, see Is llms.txt Really Unnecessary? | The Correct Reading of Google's Official Statement and Logical Gaps.
How to Write llms.txt
llms.txt is written in Markdown format. The llmstxt.org specification defines the following elements.
H1 title (required)
Start the file with your site name as an H1 heading.
blockquote for site overview (recommended)
Write a site description using a blockquote (>) block. This is what AI reads first.
H2 sections for categories
Create sections like ## Docs, ## Blog, ## Services to organize your content.
Link format for page listings
List pages as - [Page Name](https://absolute-url): Description. Relative URLs are not supported.
# Example Company > AIマーケティングプラットフォームを提供する企業です。 > コンテンツ最適化・SEO分析・パフォーマンス計測ツールを提供しています。 ## ドキュメント - [はじめに](https://example.com/docs/getting-started): セットアップガイド - [APIリファレンス](https://example.com/docs/api): エンドポイント一覧 - [料金プラン](https://example.com/pricing): プラン比較表 ## ブログ - [SEO最適化の基本](https://example.com/blog/seo-basics): 2024年最新ガイド - [構造化データの活用](https://example.com/blog/schema): Schema.org実装方法
How to Deploy llms.txt
Place the file at /llms.txt
Deploy llms.txt to your site root so it's accessible at https://yourdomain.com/llms.txt. In Next.js, placing it in /public/llms.txt serves it automatically.
Also place at /.well-known/llms.txt (recommended)
It is recommended to also place the file in /.well-known/. Some AI agents reference this path as well.
Create llms-full.txt for the full document version (optional)
llms-full.txt contains concatenated Markdown content of all pages. Useful when you want to provide detailed information to AI. Especially effective for large sites.
Common Mistakes to Avoid
Writing it in HTML
llms.txt must be written in Markdown. Using HTML tags prevents AI from parsing it correctly. Use plain Markdown only.
Missing the H1 title
The specification requires an H1 (#) with your site name at the top of the file. Without it, the file is not considered compliant with the llmstxt.org specification.
Using relative paths for links
Links must use absolute URLs like https://yourdomain.com/docs/guide, not relative paths like /docs/guide.
Not updating when the site changes
Failing to update llms.txt when pages are added or removed means you're feeding outdated information to AI. Sync it through your CI/CD pipeline or regular maintenance.
