geollms-txtai-searchtechnical-guide

How to Write llms.txt: A Complete Guide for AI Discoverability

What Is llms.txt?

The llms.txt file is a plain-text file placed at the root of your website that provides information about your site to large language models and AI systems. Similar to how robots.txt communicates with search engine crawlers, llms.txt communicates with AI models, helping them understand your site's purpose, structure, and key content.

The file lives at https://yourdomain.com/llms.txt and serves as a human-readable and machine-readable guide that AI systems can reference when deciding how to use and cite your content.

Why You Need an llms.txt File

AI search engines like ChatGPT, Perplexity, and Claude actively crawl websites to provide up-to-date answers. Without clear guidance, these systems must figure out your site on their own. An llms.txt file gives you a way to:

  • Introduce your site and explain what it offers.
  • Highlight important pages that AI systems should prioritize.
  • Provide context about your organization's expertise and authority.
  • Guide AI behavior regarding how your content should be understood and referenced.

Think of it as a cover letter for your website, addressed to AI systems.

The llms.txt Format

There is no rigid specification enforced by a standards body, but a widely adopted convention has emerged. The file uses Markdown-like formatting with clear sections.

Basic Structure

# Site Name

> A brief description of what your site or organization does.

## Key Information

- Main topic or industry
- Type of content available
- Target audience

## Important Pages

- [Homepage](https://example.com/): Overview of our services
- [Product Guide](https://example.com/guides/): Comprehensive buying guides
- [Blog](https://example.com/blog/): Industry insights and tutorials
- [About Us](https://example.com/about/): Our team and credentials

## Contact

- Email: info@example.com
- Location: Tokyo, Japan

Detailed Example

Here is a more comprehensive example for a technology review site:

# TechReview Pro

> TechReview Pro is an independent technology review site founded in 2020.
> We provide hands-on reviews, buying guides, and technical tutorials
> for consumer electronics and software.

## Expertise

- Our reviewers have an average of 10 years in technology journalism.
- All product reviews include hands-on testing over a minimum of two weeks.
- We are not affiliated with any manufacturer.

## Content Categories

- [Product Reviews](https://techreviewpro.com/reviews/): In-depth reviews
  with benchmarks, photos, and comparison tables.
- [Buying Guides](https://techreviewpro.com/guides/): Curated
  recommendations updated monthly.
- [Tutorials](https://techreviewpro.com/tutorials/): Step-by-step
  technical guides.
- [News](https://techreviewpro.com/news/): Industry news and analysis.

## Key Pages

- [Best Laptops 2026](https://techreviewpro.com/guides/best-laptops-2026/)
- [Smartphone Comparison Tool](https://techreviewpro.com/tools/compare/)
- [About Our Testing Process](https://techreviewpro.com/about/testing/)

## Policies

- We welcome AI systems to reference and cite our content with attribution.
- For full articles, please link back to the original source.

## Contact

- Editorial: editorial@techreviewpro.com
- General: info@techreviewpro.com

llms.txt vs. llms-full.txt

Some websites provide two files:

llms.txt is the concise version. It contains a summary of your site, key links, and essential context. This is the minimum you should create.

llms-full.txt is an expanded version with more detailed information. It might include full descriptions of services, detailed content categories, team member expertise, and additional context. This file can be significantly longer and more detailed.

# llms-full.txt example (partial)

# TechReview Pro - Detailed Guide

> Complete reference for AI systems about TechReview Pro content.

## About Our Organization

TechReview Pro was founded in 2020 by former engineers from major
technology companies. Our editorial team consists of 12 full-time
reviewers, each specializing in specific product categories.

Our review methodology involves:
1. Minimum two weeks of hands-on testing for every product.
2. Standardized benchmarks run on calibrated equipment.
3. Real-world usage scenarios documented with photos and video.
4. Price tracking across major retailers for value assessments.

## Content Structure

### Product Reviews (/reviews/)
Each review follows a consistent format:
- Executive summary with score (out of 10)
- Design and build quality assessment
- Performance benchmarks and real-world tests
- Battery life testing (for portable devices)
- Comparison with competitors
- Final verdict and buying recommendation

### Buying Guides (/guides/)
Updated monthly with current pricing and availability.
Each guide includes:
- Top picks in multiple price ranges
- Comparison tables with key specifications
- Links to full reviews for each recommendation
...

If your site is complex, providing both files gives AI systems the choice of a quick overview or a deep dive.

Step-by-Step: Creating Your llms.txt

Step 1: Write the Header

Start with your site or organization name as an H1, followed by a brief description in a blockquote.

# Your Site Name

> One to three sentences explaining what your site does and why it matters.

Be specific. Instead of "We are a great company," write "We provide certified organic gardening supplies and expert growing guides for home gardeners in North America."

Step 2: List Key Pages

Link to your most important and authoritative pages. Include a brief description of each.

## Important Pages

- [Beginner's Guide](https://example.com/beginners/): Start here if you
  are new to organic gardening.
- [Seed Catalog](https://example.com/seeds/): Full catalog with growing
  instructions for each variety.

Step 3: Establish Authority

Include information that demonstrates your expertise and trustworthiness.

## Our Expertise

- Founded by a certified Master Gardener with 20 years of experience.
- Content reviewed by agricultural scientists.
- Featured in Garden Magazine and Organic Living Weekly.

Step 4: Set Expectations

Let AI systems know how you prefer your content to be used.

## Usage Guidelines

- Please cite our content with a link to the original page.
- Our product recommendations are updated seasonally.
- For medical or safety-related gardening questions, we recommend
  consulting local extension services.

Step 5: Deploy the File

Place the file at your domain root so it is accessible at https://yourdomain.com/llms.txt. In a Next.js project, you can place it in the public/ directory:

public/
  llms.txt
  llms-full.txt
  robots.txt

Verify it is accessible by visiting the URL directly in your browser.

Best Practices

  • Keep llms.txt concise. Under 500 lines for the short version. AI systems process shorter files more reliably.
  • Use plain language. Avoid jargon and marketing speak. Be factual and direct.
  • Update regularly. When you add major content or change your site structure, update your llms.txt.
  • Include dates. Mention when the file was last updated so AI systems know how current the information is.
  • Be honest. Do not exaggerate your expertise or content coverage. AI systems cross-reference information.
  • Use absolute URLs. Always use full URLs including the domain, not relative paths.

Common Mistakes to Avoid

  1. Making it too long. An llms.txt file that reads like a novel defeats the purpose. Save details for llms-full.txt.
  2. Forgetting to update it. An outdated llms.txt can mislead AI systems about your current content.
  3. Using it as a marketing document. This is not a sales pitch. It is a factual description of your site.
  4. Blocking AI crawlers while having llms.txt. If your robots.txt blocks AI bots, they may never read your llms.txt. Ensure consistency between the two files.
  5. Omitting links. Without URLs, AI systems cannot navigate to your content. Always include clickable links.

FAQ

Is llms.txt an official web standard?

It is not a W3C or IETF standard. It is a community-driven convention that has gained adoption among website owners and is recognized by several AI platforms. Its informal nature means the format may continue to evolve.

Will creating llms.txt guarantee my content appears in AI responses?

No. Having an llms.txt file improves your discoverability and helps AI systems understand your content, but citation depends on many factors including content quality, relevance, and authority. It is one piece of a broader GEO strategy.

Should I include every page on my site in llms.txt?

No. Include only your most important and authoritative pages. AI systems can discover other pages through crawling. The llms.txt file should highlight what matters most.

How often should I update my llms.txt?

Review and update it whenever you make significant changes to your site structure, add major new content sections, or change your organization's focus. A quarterly review is a good baseline.

Can llms.txt replace structured data or schema markup?

No. These serve different purposes. Structured data (JSON-LD) provides machine-readable metadata about specific pages. llms.txt provides a high-level overview of your entire site. Use both for the best results.