A growing share of the questions that used to end in a list of blue links now end in a written answer with three or four sources attached. Someone asks an assistant for the best invoicing tool for freelancers, or a plumber in their city, and gets a paragraph naming a handful of options. If your site isn't among the sources, you were never in the conversation — and you won't see a single impression to tell you what happened.
The key takeaway: AI answers are assembled from pages that are crawlable, unambiguous about what they describe, and already named alongside your competitors elsewhere on the web. That last part is the one most site owners skip. You cannot make an assistant mention you, but you can make yourself easy to find, easy to parse, and hard to leave out of a shortlist.
Here is what that actually involves.
How an AI answer gets built
Understanding the pipeline tells you where to spend effort. Most assistants that cite sources do roughly this:
- Retrieve. It runs one or several searches — sometimes against its own index, sometimes a partner search index — to pull candidate documents.
- Read. It fetches those pages and extracts the passages that address the question.
- Synthesise and attribute. It writes an answer and links the pages that carried the load.
Two consequences fall out of that. Retrieval is still search: if your page isn't in an index somewhere, everything downstream is moot. And the reading step rewards pages that state facts plainly in text, not pages hiding them in an image or a script that runs after a click.
If your site isn't reliably indexed yet, that's the prerequisite, not a parallel project — our online visibility guide covers the groundwork.
Check that AI crawlers can actually reach you
This is the quiet, common failure. Plenty of sites added blanket blocks for "AI bots" — sometimes deliberately, often because a plugin or CDN offered a one-click toggle — then wondered why they never appear in AI answers.
The thing to understand is that these companies run more than one crawler, with different jobs, and blocking the wrong one has effects you didn't intend:
| User agent | What it is for | Blocking it means |
|---|---|---|
GPTBot |
OpenAI's crawler for model training | Your content isn't used for training; search citation is a separate matter |
OAI-SearchBot |
OpenAI's crawler for surfacing sites in ChatGPT search | You opt out of being cited there |
ChatGPT-User |
Fetches a page when a user's request requires it | Live lookups of your pages fail |
PerplexityBot |
Perplexity's indexing crawler | You opt out of Perplexity's citations |
Google-Extended |
Controls use of your content for Google's generative AI products | Does not remove you from Google Search |
Googlebot |
Ordinary Google crawling and indexing | You disappear from Google Search entirely |
Open your robots.txt and read it line by line — a User-agent: * block with a broad Disallow catches everything. If your goal is visibility rather than content protection, allow the search-oriented crawlers explicitly. Check at the CDN level too: some bot-management rules block by user agent before the request reaches your server, quietly overruling a permissive robots.txt.
Then verify rather than assume. Your access logs show which agents actually fetched pages this month. If none have, you have a reachability problem, not a content problem.
Write pages a machine can quote
Retrieval and extraction both favour a specific shape of page. None of this is exotic; it is the same clarity that helps human readers skim.
- Answer the question in the first two sentences of a section. Extraction pulls passages, not whole pages. Three paragraphs of throat-clearing gives the system nothing clean to lift.
- Use headings that match real questions. "How much does it cost?" beats "Investment." Literal phrasing helps retrieval and readers alike.
- Put facts in text, not in graphics. Prices, hours, service areas, and comparisons that live only inside an image are invisible to extraction. Repeat them in the copy.
- Make key facts self-contained. "We serve the whole metro area" is useless out of context; "We serve Portland, Oregon and surrounding suburbs" survives being lifted into an answer.
- Render content server-side where you can. If the substance only appears after JavaScript runs, some crawlers see an empty shell. Check the raw source, not the rendered DOM.
Structured data reinforces the same clarity. Marking up an Organization, LocalBusiness, or Product states unambiguously what an entity is — name, location, relationships — rather than leaving it to be inferred from prose. It removes ambiguity, and ambiguity is what gets you dropped from a shortlist.
Why directory listings matter more here than in classic SEO
Here is the pattern worth internalising: when an assistant is asked to name options — best tools for X, top providers in Y, alternatives to Z — the pages it retrieves are overwhelmingly pages that already contain lists of options. Curated directories, category rankings, comparison pages, and "best of" roundups are the natural source material for a question shaped as "which should I pick?", because they are the only documents on the web that already answer it in that form.
Your own homepage will never be retrieved for "best project management tools for small agencies," no matter how good your product is. It contains exactly one option. The pages that get read for that query are the ones listing twenty.
That gives you a concrete job: be present in the lists that already rank for the phrases your buyers use. Practically, that means:
- Identify the category pages and directory rankings that surface for your core commercial queries. Search a few of them yourself and note which listing sites appear repeatedly.
- Get listed on the ones that are genuinely curated and relevant to your niche — quality and topical fit, not volume. Our guide to the best business directories explains how to judge which ones earn the effort.
- Keep the entry accurate and complete. A stub listing with a name and a URL gives an extraction step nothing to say about you; a listing with a real description, category, and consistent details gives it a sentence it can reuse.
Consistency matters more here than in classic citation-building, because a machine is reconciling mentions of you across dozens of sources. A business named and described identically in fifteen places reads as one confident entity; one appearing under three slightly different names reads as noise.
Own your brand answer first
Before chasing category queries, check what assistants say when asked directly about you. Ask two or three of them to describe your business, and note what comes back: right, wrong, or nothing at all.
Wrong answers usually trace to thin or conflicting source material — an outdated listing, a stale profile, a description you changed on your site but nowhere else. Nothing at all usually means there's almost no third-party presence to synthesise from. Both are fixed the same way: a clear, factual about page, accurate Organization markup, and consistent listings across the profiles that describe you. The same forces apply in ordinary search, which we cover in why your business doesn't show up when you Google its name.
What about llms.txt?
You will see this recommended confidently. llms.txt is a proposed convention — a Markdown file at your site root offering a curated map of your content for language models to read. It is a sensible idea, costs an hour, and breaks nothing.
Be honest about its status, though: it remains a community proposal, not something the major AI providers have committed to consuming. Treating it as your AI visibility strategy, in place of crawlable pages and real listings, would be a mistake. Do the durable work first.
Measuring it (imperfectly)
There is no Search Console for AI answers, so measurement is indirect. Watch three things: referral traffic from assistants that link out, segmented in your analytics; manual spot checks, running ten questions your buyers actually ask across two or three assistants monthly and recording whether you're mentioned; and server logs, where rising fetches from search-oriented AI crawlers are the leading indicator that your content is being read.
Expect variance. The same question asked twice can return different sources. Judge direction over months, not answers on a Tuesday.
FAQ
Does blocking GPTBot stop me appearing in ChatGPT answers?
Not by itself. GPTBot and the search-oriented crawler are separate agents with separate purposes, so blocking the training crawler while allowing the search one is a coherent position. Check which agents your robots.txt and your CDN rules actually cover.
Does structured data get me into AI answers? It does not force inclusion. What it does is state plainly what your page and your organisation are, so a machine doesn't have to guess. That reduces the chance of being misdescribed or skipped, which is the realistic benefit.
Is this different from ordinary SEO? The foundations are the same — crawlable, indexed, clearly written pages. The emphasis shifts: passage-level clarity and third-party mentions matter more, and being present in list-shaped pages matters much more, because those are what get retrieved for "which one should I choose" questions.
How long does it take to see a change? Slower than ordinary indexing, because your listings and mentions have to be crawled and re-read before they influence anything. Treat it as a quarter-scale project, not a week.
Where to start this week
Four things, in order: confirm your robots.txt and CDN rules allow search-oriented AI crawlers; rewrite your three most commercially important pages so each section answers its question in the first two sentences; add Organization markup with your exact name, location, and category; then find the curated category lists ranking for your core query and get into them accurately.
That last step has the longest tail. When someone asks an assistant to name the best options in your category, it reads the pages that already name options. Add your website to the ranking that matches your niche so that when the shortlist gets read, you are on it.
