How to get indexed by LLM through llms.txt file?

You don’t “submit” your llms.txt file to AI models like you submit a sitemap to Google. Instead, when an AI agent (like ChatGPT or Perplexity) is asked a question related to your brand or industry, it may crawl your site in real-time. If it finds an llms.txt file at your root domain, it will use that file as a roadmap to quickly understand your content.

Is llms.txt for online stores necessary?

Yes. For an online store, AI is quickly becoming a real discovery channel alongside Google. An llms.txt file for your online store means that when AI recommends products, compares suppliers, or answers questions about your brand, it’s working from accurate, structured information you control, not a guess pieced together from scraped HTML or a competitor’s marketing copy. It won’t replace product SEO or your sitemap, but it closes the gap between what a shopper’s AI assistant sees and what your store’s pages actually say.

What is the difference between llms.txt and agents.md?

llms.txt is a descriptive summary of your site’s content, designed to give AI models context. agents.md is a set of instructions specifically for autonomous AI agents, telling them how to interact with your site, find APIs, or complete transactions (like adding items to a cart) .

How often should I update my llms.txt file?

It’s recommended to update your llms.txt file whenever there are significant changes to your website’s core content, product catalog, or services. This ensures AI models always have access to the most current and relevant information. For dynamic sites, consider automating the generation of this file.

Can llms.txt replace robots.txt or sitemap.xml?

No, llms.txt serves a different purpose and does not replace robots.txt or sitemap.xml. robots.txt controls crawler access, sitemap.xml lists all pages for indexing, and llms.txt provides a curated, AI-friendly summary of your most valuable content. All three files play distinct and important roles in your website’s discoverability.

What happens if I don't implement llms.txt?

If you don’t implement llms.txt, AI models will still attempt to understand your website by crawling and parsing its content. However, without a structured llms.txt file, they may struggle to identify your most important content, leading to less accurate AI-generated answers or missed opportunities for AI-powered discovery.

Are there any tools to help generate llms.txt?

Yes, for platforms like WordPress, there are plugins such as “Website LLMs.txt” and “LLMs.txt and LLMs-Full.txt Generator” that can automate the creation and management of these files. For other platforms, you might need to manually create the Markdown file or use custom scripts to generate it dynamically.

LLMS.txt Setup Guide For Shopify, WordPress And Magento

Introduction

The online world is changing at breakneck speed. People don’t just rely on search engines any more AI-driven Large Language Models (LLMs) like ChatGPT, Gemini, and Claude are quickly becoming the way users discover, reference, and interact with information.

For forward-thinking businesses, this isn’t just another digital trend. AI brings extraordinary opportunities for exposure, but it also introduces fresh questions about who owns your content, how your brand is perceived, and how you keep your digital assets safe.

Enter LLMS.txt your new line of defence.

At Kiwi Commerce, we help ambitious brands take control of their online presence in a landscape that’s always shifting. This guide will walk you through exactly what LLMS.txt is, why it’s essential, and step-by-step instructions for setting it up on platforms like WordPress, Shopify, Magento, and custom-built sites.

What is LLMS.txt and Why Does It Matter?

Think of LLMS.txt as the modern cousin to robots.txt. Where robots.txt tells search engine spiders which pages to crawl or ignore, LLMS.txt speaks to AI bots those LLM-powered tools that are busy gathering and learning from web content, sometimes without you even realising.

So, what can you do with LLMS.txt?

Decide if AI crawlers can see all, some, or none of your website.
Write custom rules for individual bots like letting ClaudeBot in, but keeping GPTBot out.
Shield private or sensitive parts of your site, like user accounts or checkout pages, from being scraped.
Make sure your content appears in AI outputs the way you intend or is left out altogether.

Why Should Every Business Have LLMS.txt?

Let’s be honest: your website content is one of your most valuable digital assets. With LLMS.txt in place, you can…

Safeguard your hard work – Stop competitors, aggregators, or rogue bots from misusing your product descriptions, blog posts, and creative assets.
Protect your brand’s reputation – Prevent out-of-context AI summaries from misrepresenting your tone, ethics, or expertise.
Boost data privacy – Keep customer and sensitive business info away from prying digital eyes.
Stay ahead of the curve – Show your customers and partners that you take AI compliance and innovation seriously.

For us at Kiwi Commerce, adding LLMS.txt isn’t just about following best practice it’s about owning your brand’s story in an AI-powered world.

How To Add LLMS.txt to Your Website

The nuts and bolts depend on what your site is built with. Here’s how to do it for each major system:

WordPress

Log into your web hosting panel (like cPanel or Plesk), or use an FTP client.
Navigate to your site’s root folder, usually called /public_html/.
Create a new text file, and call it llms.txt.
Add your crawler rules. Try this as a template:

User-agent: GPTBot

Disallow: /wp-admin/

Allow: /blog/

Save and upload the file. Test it’s live by visiting www.yourdomain.com/llms.txt.

Kiwi Commerce tip: If you prefer, use a plugin such as File Manager or an SEO tool that lets you edit files directly in your dashboard no coding necessary.

Shopify

Shopify doesn’t allow direct access to your website’s root, but there is a workaround:

In your Shopify admin, go to Online Store → Themes.
Click Edit Code on your active theme.
In the Assets folder, create a new file called llms.txt.
Add your rules. For example:

User-agent: *

Disallow: /checkout/

Allow: /collections/

Save, then check www.yourdomain.com/llms.txt to make sure it’s accessible.

Kiwi Commerce tip: Want even tighter control? We offer developer solutions to proxy the file properly, so even advanced bots get the message.

Magento

Connect to your server via FTP or SSH.
Head to your Magento root directory.
Create llms.txt and include rules such as:

User-agent: ClaudeBot

Disallow: /customer/

Disallow: /checkout/

Allow: /products/

Upload and make sure it’s live at www.yourdomain.com/llms.txt.

Kiwi Commerce tip: Always block customer and checkout pages—keep that sensitive data safe.

Custom-Coded Sites (PHP, Node.js, React, etc.)

Open your site’s root directory.
Create a new text file—llms.txt.
Enter your rules, like:

User-agent: *

Disallow: /internal/

Allow: /articles/

Deploy the file to your live web server. Double-check it loads via your browser.

Kiwi Commerce tip: Ensure your server (Apache, Nginx, etc.) is set up to serve .txt files. If you’re not sure, ask your hosting provider most can help in minutes.

LLMS.txt Best Practices

Always put the file at your site’s root for maximum effect.
Write clear, simple rules avoid technical jargon or vague language.
Update the file as your website grows, adding or removing rules as needed.
Block all private or sensitive areas by default. Allow areas specifically designed for public or marketing content.
Remember that LLMS.txt is advice to AI bots it’s not bulletproof security, so always use it as part of a broader digital protection plan.

Example LLMS.txt Configurations

Block All AI Bots

User-agent: *

Disallow: /

Allow Only Your Blog

User-agent: *

Disallow: /

Allow: /blog/

Block Just One AI Bot (e.g., GPTBot)

User-agent: GPTBot

Disallow: /

Final Thoughts

Option 1: Manual (via FTP or File Manager)

Create a new plain text file on your computer and name it llms.txt.
Write your content in Markdown format (see the example above).
Connect to your server via FTP (using a client like FileZilla) or use your hosting panel’s File Manager.
Upload the file to your site’s root directory—the same folder where wp-config.php lives, typically /public_html/.
Verify it is live by visiting https://yourdomain.com/llms.txt in your browser.

Option 2: Free Plugin

If you would rather not work with files directly, the “Website LLMs.txt” plugin (available free on WordPress.org ) handles file generation and management from within your WordPress dashboard. It lets you choose which post types and pages to include, and it generates a properly formatted Markdown file automatically.

Another solid free option is the “LLMs.txt and LLMs-Full.txt Generator” plugin, which generates both the summary llms.txt and a comprehensive llms-full.txt in one go.

Kiwi Commerce tip: If you already use Yoast SEO, it has a basic llms.txt feature built in, though it generates a fairly minimal file. The dedicated plugins above give you more control over what is included.

AI is reshaping how people discover brands and products. With LLMS.txt, you call the shots deciding exactly which areas of your website should be open to AI crawlers and what should remain off-limits.

Here at Kiwi Commerce, we’re experts at integrating LLMS.txt across WordPress, Shopify, Magento, and custom sites. Whether you want your website invisible to LLMs or you’d like to showcase certain pages for digital visibility, we ensure your content and your brand is protected.

Future-proof your website today. Contact KiwiCommerce to get started with LLMS.txt and secure your business for tomorrow’s digital challenges.

Article written by

Sam

Head of E-commerce
Having worked with online shops for more than 8 years, he is an eCommerce SEO expert who helps them boost organic traffic and sales. He specialises in digital PR, content planning, and technical SEO. He has increased organic revenue by more than 3 times and assisted in the recovery of several websites from traffic fines.

LLMS.txt: Setup Guide for WordPress, Shopify, Magento, and Custom Websites

Introduction

What is LLMS.txt and Why Does It Matter?

Why Should Every Business Have LLMS.txt?