- October 23, 2026
- Artificial Intelligence, SEO
Generative Engine Optimization in 2026: what Google, Bing and OpenAI actually said
2026 is the year generative engine optimization stopped being guesswork. Google published an actual AI optimization guide, Bing started showing citations, and every major AI company split its crawlers by job. Here is every official announcement that matters, dated and linked to the source, plus what each one does (and doesn't) mean for your site.
Key takeaways
- Google says GEO is SEO. Its May 15 guide covers GEO and AEO, explains query fan-out and grounding, and rejects special AI schema, AI-specific wording and mandatory tiny chunks.
- You can finally measure some of it. Google reports AI Overviews and AI Mode impressions; Bing reports citations, Citation Share and topic opportunities. Neither one counts clicks or sales for you.
- There's now a dedicated AI switch. Google's Search generative AI control (Search Console) manages inclusion in AI Overviews and AI Mode without touching the rest of Search or AI training.
- Search access is not training permission. OAI-SearchBot vs GPTBot, Claude-SearchBot vs ClaudeBot. Block training if you want; you don't lose search eligibility for it.
- llms.txt does nothing for Google Search. It may help other agents if they fetch it, which most currently don't.
- A citation is not a visit, and a visit is not a sale. Track each stage of the chain separately.
5 interactive tools inside this guide
Not every "announcement" is a ranking rule
Half the GEO advice floating around LinkedIn blurs a vendor study into a Google requirement. Every item below is tagged with one of four evidence types. Click a type to filter the timeline to it.
Official statements establish what a company says. Company claims about traffic, conversions or improvements are not independently verified outcomes. An undated help page shows current documented behavior, not a new announcement date.
The three shifts that actually change your work
Google went on the record
An explicit AI optimization guide, dedicated generative AI reporting and a separate Search generative AI inclusion control, all live worldwide by August 31.
Microsoft opened the citation box
AI Performance in Bing Webmaster Tools now shows citations, Citation Share, intents and a Topics Portfolio of opportunity queries.
Access, training and outcomes split apart
Search access, training permission, answer visibility, clicks and sales are now separate dials, each with its own controls and measurements.
Ten months of GEO announcements on one map
Each quarter gets its own lane. Press play to watch 2026 unfold, hover or tap any dot for the headline, and click a company name or month to filter the timeline below.
Dated items per month, 2026
October is the busiest month so far, with nine items in three weeks. Click a bar to filter.
Who's driving it
All 36 items by organization (the Cloudflare and OpenAI pilot counts for both). Click to filter.
Every official GEO update in 2026
Search, filter by company or evidence type, and flip the order. Every entry links to its primary source, and "Show on map" jumps back to the dot above.
October 2026
- 30
AhrefsVendor researchAhrefs explains AI share of voice
Vendor measurement guidance
AI share of voice depends on prompts, platforms, competitors, location, period and dataset. Brand Radar estimated impressions use modeled weighting and are not a census of people seeing answers. Visibility is separate from sentiment, recommendations, traffic and conversion.
What it means: Disclose the denominator and collection method before comparing tools or reporting a lift.
Source: Ahrefs' measurement article
- 31
AhrefsVendor opinionAhrefs on "headless" content marketing
Attributed industry interpretation
Ryan Law describes content made for retrieval and synthesis through AI intermediaries, argues changing distribution incentives explain it, and admits its durability is uncertain.
What it means: A significant industry argument, not Google approval for machine-first pages. Compare it to the target engine's policies and to real business value.
Source: Ahrefs' interpretation
- 1
GooglePlatform guidanceGoogle updates its content-sharing controls documentation
Documentation clarification
The overview now lists robots metadata,
data-nosnippet,X-Robots-Tagand the Search generative AI control side by side. Google describes that last one as a site-wide way to manage inclusion in AI Overviews and AI Mode through Search Console. This is a documentation update, not a fresh launch of every control on the list.What it means: Your technical GEO checklist should separate three things: whole-search exclusion, snippet restrictions and Search AI inclusion.
Source: Google documentation Dated changelog
- 22
OpenAICloudflareVendor researchCloudflare and OpenAI discovery pilot reports early results
Joint research pilot
The pilot began in July. A cohort representing about 1% of Cloudflare's network enabled discovery and indexing of hundreds of millions of previously unknown pages. Cloudflare supplies discovery signals; OpenAI independently evaluates relevance and ranking. Business-outcome work is still planned.
What it means: Discoverability is getting direct infrastructure investment. The result is about indexing, not proven gains in citations, conversions or revenue, and not every Cloudflare site participates.
Source: Joint pilot results
- 16
Microsoft / BingProduct announcementTopics Portfolio beta in AI Performance
Bing Webmaster Tools feature (beta)
Groups grounding queries into themes with approximate topic size, number of sites cited, query counts and opportunity queries drawn from your own AI Search activity. Smaller sites may not see it. Microsoft calls the numbers directional: classification can be imperfect, opportunity counts use ranges, and branded queries can show up as opportunities.
What it means: Handy for topic planning. Check evidence and business relevance before you chase a suggested gap.
Source: Bing's announcement
- 32
SemrushVendor researchSemrush compares UK and US AI visibility
Vendor observational research
January to July 2026 data, using fixed monthly sets of 2.5 million UK and 25 million US prompts for ChatGPT, Gemini and AI Mode, plus a variable AI Overviews keyword set. Mentioned brands and cited sources differ by market and platform.
What it means: Keep geographic and platform benchmarks separate. A huge sample does not make findings universal or prove causal ranking factors.
Source: Semrush's research release
- 33
AhrefsVendor researchAhrefs studies ChatGPT query fan-out
Vendor study (with correction)
66.6 million free, logged-out responses from August and September, plus 15 million associated fan-out queries. The corrected finding: 14.4% contained fan-out queries. That is not the same as saying only 14.4% searched the web.
What it means: Don't repeat the corrected-away claim that 85.6% never searched. Keep product facts accessible and watch the answer modes your audience uses.
Source: Ahrefs study and correction
- 2
GoogleProduct announcementUGC Fresh Data Program documentation appears
Specialized ingestion program
Approved high-volume UGC platforms can submit fresh content and interaction signals through a dedicated pipeline. Conditions include public accessibility, creator attribution, moderation, structured data and technical capacity. It sits apart from ordinary crawling and the Indexing API, and appearance is not guaranteed.
What it means: Relevant for qualifying forums and social platforms. It is not a submission shortcut for every business site, and it is not proof that buying forum mentions improves AI visibility.
Source: Google program documentation
- 3
GooglePlatform guidanceAI-generated content guidance is updated
Content-quality guidance
Google connected its AI content guidance to the Search Quality Raters Guidelines, including scaled content abuse and material with little effort, originality or added value. It calls for human accuracy checks (metadata included) and notes that rater judgments do not directly determine rankings.
What it means: AI assistance and useful publishing can coexist. Bulk production with no added value is a separate problem. There is no blanket penalty for AI-written text in this update.
Source: Google's AI content guidance
September 2026
- 28
CloudflareProduct announcementCloudflare Pay Per Use enters beta
Publisher monetization infrastructure
AI companies define a paid downstream use and an offer, publishers choose whether to participate, and Cloudflare handles usage reporting and payments. Cloudflare separates this from its earlier Pay Per Crawl access model.
What it means: A crawl, a citation and a compensated use are three different events. Joining does not promise payment from every AI operator or better rankings.
Source: Cloudflare's beta announcement
- 4
GoogleProduct announcementMultimodal Search reporting lands in Search Console
Reporting rollout (global)
A web multimodal search filter now appears in both standard Search performance and generative AI performance reporting. It covers Lens, Circle to Search, image uploads and Chrome image search. Data shows up once your site gets that kind of activity.
What it means: AI discovery measurement now includes visual queries. If you only monitor text prompts, you are missing part of the picture.
Source: Search Central announcement
June 2026
- 34
Adobe / SemrushProduct announcementAdobe Brand Visibility launches
Commercial GEO product
Combines Semrush AI visibility intelligence with Adobe optimization and analytics: prompt research, competitive visibility, deployment and links to business outcomes.
What it means: Confirms where commercial tooling is heading. Customer results need their own methodology review; this is not an engine-endorsed technique.
Source: Semrush / Adobe announcement
- 17
Microsoft / BingProduct announcementIntents, Topics, Citation Share and Compare
Expanded AI Performance (global preview)
Microsoft added intent categories, thematic clustering, time comparisons and Citation Share: your site's percentage of all citations for a specific grounding query. Microsoft says it is observational, does not reveal competitor domains, and is not traffic share or a quality score.
What it means: Citation counts and citation share answer different questions. Neither one counts visits or purchases.
- 7
GooglePlatform guidanceGoogle states its position on llms.txt
Explicit documentation clarification
The changelog says maintaining llms.txt files for other services is fine, and that they neither improve nor damage Google Search visibility or rankings.
What it means: Any claimed llms.txt benefit needs a label: which system consumes it, and what outcome was measured.
Source: Google's dated clarification
- 35
AhrefsVendor researchAhrefs studies llms.txt access
Vendor log study
Of roughly 38,000 valid files found across 137,210 Ahrefs Web Analytics domains, 97% received no requests in May 2026. The population skews technical and SEO-aware. The study measures a root index file and requests, not page-level Markdown delivery or citation lift.
What it means: Test whether the consumers you care about fetch and use the file. Don't stretch this into proof that every alternate format is useless.
Source: Ahrefs' study and methodology
- 8
GooglePlatform guidanceGuidance on third-party SEO and GEO tools
Official evaluation guidance
Google tells site owners to check outside advice against official documentation. Third-party services are not evaluated or approved by Google, have no access to its internal ranking data, and cannot guarantee performance. Search Console remains the first-party source.
What it means: An "AI visibility score" is a tool's measurement, not a Google quality score. Ask for transparent evidence any time a claim is attributed to Google.
- 5
GoogleProduct announcementGenerative AI performance reports for Search and Discover
First-party measurement product
Dedicated reports for generative AI features list impressions, pages, countries, dates and devices for Search. The Search report covers AI Overviews and AI Mode, excludes Search Labs experiments, and may not appear when impressions are too low. Worldwide availability was confirmed on August 31.
What it means: Retire the line that Google offers no separate AI visibility reporting. It documents impressions, not a prompt-level, click, conversion or revenue dashboard, so measure downstream outcomes elsewhere.
- 6
GoogleProduct announcementSearch generative AI control rolls out worldwide
Publisher inclusion control
A separate control decides whether your site's links and content can appear in and ground generative Search responses. The test started with a UK subset before the worldwide rollout. Help docs cover include, exclude and inherit-from-parent. Inclusion is the default for top-level properties. It does not decide ranking elsewhere in Search, does not override merchant or ads participation, and does not control AI training.
What it means: When AI visibility goes missing, check the property and its inherited setting first. Handle training controls as a separate decision.
May 2026
- 9
GoogleProduct announcementPreferred Sources expands into AI Overviews and AI Mode
Personalized source presentation
Preferred badges can appear in supported AI Mode and AI Overviews locales for users who picked that source. Sites need Search AI inclusion turned on. Domain and subdomain properties qualify; subdirectories do not. An August 20 update added an interactive button implementation.
What it means: This is a direct audience relationship feature, not a universal ranking boost.
- 10
GoogleProduct announcementGoogle announces a new AI Search experience and agents
Product direction
Elizabeth Reid describes an intelligent Search box and new agent capabilities. Search is moving toward getting tasks done, not just finding information.
What it means: Being retrieved for an answer and enabling a booking or purchase are different wins. A product announcement does not reveal a new ranking formula.
Source: Google's I/O Search announcement
- 11
GooglePlatform guidanceGoogle publishes an explicit generative AI optimization guide
Central platform guidance
Announced by John Mueller, the guide places GEO and AEO inside SEO and explains retrieval grounding and query fan-out. It prioritizes original, experienced, useful content and ordinary technical eligibility. It rejects mandatory tiny content chunks, AI-specific wording, special AI schema, and any Google Search benefit from special AI files or Markdown. It also warns against manufacturing pages for every fan-out variation and chasing inauthentic mentions.
What it means: Teach useful content and reliable access first. Google does not endorse a universal GEO hack list.
- 12
GoogleProduct announcementGoogle changes how AI answers link to the web
Answer-interface update
Five changes: further-reading suggestions, subscription labels, previews of firsthand and community perspectives, more inline links, and desktop link previews.
What it means: Traffic shifts can come from presentation changes, not just source selection. Authentic third-party discussion matters; fabricated mentions are not approved.
Source: Google's five announced changes
- 13
GooglePlatform guidanceChrome Lighthouse adds an llms.txt agentic audit
Browser / agent audit docs
Chrome documents an agentic browsing check for
llms.txt. A server error gets flagged; a missing file returning 404 is marked not applicable because the file is optional.What it means: Chrome acknowledges possible agent navigation uses. That is a different system from Google Search, and it does not make llms.txt a Search visibility signal.
Source: Chrome's audit documentation
April 2026
- 25
AnthropicPlatform guidanceAnthropic documents three crawler roles
Current crawler documentation
ClaudeBothandles potential training collection,Claude-SearchBothandles search, andClaude-Userhandles user-directed access. Disabling the search or user agents can reduce search visibility or retrieval, and Anthropic documents respect for robots directives. The displayed date is not proof all three launched that day.What it means: Set policies for model training, search discovery and user-requested retrieval one at a time.
Source: Anthropic's crawler guidance
March 2026
- 36
YoastProduct announcementYoast adds llms.txt to its Shopify app
Vendor feature launch
An opt-in feature builds a store guide from selected products, collections, policies and pages, with automatic or manual selection and weekly updates.
What it means: Tools make an agent-facing file easier to maintain. That does not prove Google Search rewards it; usefulness depends on a system actually fetching it.
Source: Yoast's announcement
- 23
OpenAIProduct announcementOpenAI expands product discovery through ACP
Shopping and product-data integration
The Agentic Commerce Protocol now supports product feeds and promotions, with richer product comparison. Shopify Catalog already supplies merchant data, and the focus shifts toward discovery with merchants' own checkout.
What it means: Product feeds are their own discovery route. Feed participation does not guarantee organic recommendations, and older Instant Checkout descriptions need a date check.
- 27
PerplexityProduct announcementPerplexity adds professional Premium Sources
Source partnership
Selected Statista, CB Insights and PitchBook material is available inside Computer research, with citations and source links.
What it means: Source supply now includes licensed integrations. Public-web optimization can't replicate a negotiated data partnership.
February 2026
- 18
Microsoft / BingPlatform guidanceMicrosoft frames GEO around grounding
Executive platform statement
Jordi Ribas describes grounding as the link between AI and current, authoritative information, and frames GEO as participation in answers, citations, reasoning and outcomes. The new Webmaster Tools reporting is positioned as an early window into that.
What it means: Microsoft uses GEO openly as an operating and measurement concept. That does not mean crawling and indexing stopped mattering.
Source: Microsoft's grounding statement
- 29
CloudflareProduct announcementCloudflare launches Markdown for Agents
Content delivery feature
Enabled sites answer an
Accept: text/markdownrequest with a converted Markdown version of the page. Cloudflare shows substantially lower token usage and reports compatible coding agents requesting the format.What it means: A real way to serve page substance to agents. It demonstrates delivery and efficiency, not a controlled lift in AI Search rankings, and it is a different thing from an llms.txt link index.
- 19
Microsoft / BingProduct announcementAI Performance public preview in Bing Webmaster Tools
First-party citation reporting
Covers Microsoft Copilot, Bing's AI-generated summaries and selected partner integrations. Shows total citations, average cited pages, sampled grounding queries, URL activity and trends. Grounding queries summarize activity rather than reveal exact prompts, and changes cannot be pinned to one cause automatically.
What it means: Real platform data, but its coverage is Microsoft's surfaces and partners, not the whole AI ecosystem.
January 2026
- 14
GoogleProduct announcementUniversal Commerce Protocol, Business Agent and richer merchant data
Commerce infrastructure
Google announced the Universal Commerce Protocol, a branded Business Agent experience in Search, and new Merchant Center attributes for conversational discovery (product questions, compatible accessories, substitutes). Eligibility and geography vary; Business Agent launched for eligible US retailers.
What it means: Commerce visibility now runs through accurate catalogs and merchant integrations, not just editorial pages. The same announcement includes Direct Offers advertising, which is not an organic citation.
Source: Google's retailer announcement
Standing documentation (current pages, no new publication date asserted)
- 20
Microsoft / BingPlatform guidanceBing Webmaster Guidelines explicitly cover GEO
Official platform policy
The guidelines tie discovery, indexing, authority, clear entities, verifiable facts, useful structure and freshness to grounding eligibility. They recommend IndexNow, accurate sitemaps and crawlable links, and warn against prompt injection, artificial citation manipulation, misleading markup and low-value scaled generation. They say plainly that GEO does not guarantee grounding or citations. Exact revision date not established.
What it means: Keep your technical SEO and check answer eligibility and abuse policies. Don't repeat an update date from secondary coverage as verified.
Source: Bing Webmaster Guidelines
- 24
OpenAIPlatform guidanceOpenAI separates search, training and user fetching
Technical crawler policy
OAI-SearchBotsupports ChatGPT search,GPTBotconcerns potential training use, andChatGPT-Userhandles some user-directed visits. Search and training preferences are independent. OpenAI recommends allowing its search bot and published IP ranges. Search opt-outs may still appear as navigational links.What it means: You do not have to allow training to participate in search. Debug access for the right agent instead of treating every OpenAI request the same.
Source: OpenAI crawler documentation
- 26
PerplexityPlatform guidancePerplexity's search and user agents
Technical crawler policy
PerplexityBotsurfaces and links sites in search and is not a foundation-model training crawler; Perplexity recommends allowing it and its published IPs.Perplexity-Usersupports requested visits and, per the docs, generally ignores robots.txt because the request is user initiated.What it means: Robots rules, firewall settings and user-agent roles all matter, and fetching policies differ between AI companies.
Source: Perplexity crawler documentation
2025 foundations (still useful baselines)
- 15
GooglePlatform guidanceGoogle's 2025 advice for succeeding in AI Search
Earlier official baseline
Unique content, a satisfying visitor experience, technical accessibility and useful images and video.
What it means: 2026 added documentation, controls and measurement. It did not erase the focus on human usefulness.
Source: Google's 2025 advice
- 21
Microsoft / BingPlatform guidanceMicrosoft's practical advice for AI answer inclusion
Microsoft-hosted practitioner guidance
Krishna Madhavan recommends descriptive headings, clear facts, self-contained answers, relevant lists and tables, and accurate schema, and discourages burying essential information in hard-to-access formats.
What it means: Supports clear structure for Microsoft experiences. It is not a Google rule for passage length, FAQ format or special schema.
Source: Microsoft's content advice
No updates match those filters.
GEO vs SEO: what Google and Bing actually said
The "GEO vs SEO" and "AEO vs GEO" debates got a lot simpler this year. Google's May 15 guide treats GEO and AEO as part of SEO. Microsoft uses GEO as its own operating term, but its guidelines still start with discovery, indexing and crawlable links. If you want the visual version, my SEO, AEO and GEO quick reference infographic lays out the shared foundations.
SEO is the foundation
Crawling, indexing, technical eligibility and useful content. Google's May 15 guide puts GEO and AEO inside it rather than beside it.
What Google's guide prioritizes
- Original, experienced, useful content
- Ordinary technical eligibility: crawlable, indexable, snippet-eligible
- Understanding retrieval grounding and query fan-out
- Authentic third-party discussion about you
- Useful images and video (carried over from the 2025 baseline)
What Google's guide rejects
- Mandatory tiny content chunks
- AI-specific wording
- Special "AI schema"
- Google Search benefits from special AI files or Markdown
- Manufacturing a page for every fan-out variation
- Seeking inauthentic mentions
Microsoft's own 2025 practitioner advice does favor descriptive headings, self-contained answers, lists, tables and accurate schema for its experiences (item 21). Use it for Bing and Copilot. Just don't present it as a Google requirement. For the full roadmap, see my GEO deep dive infographic.
The AI visibility output chain
Improvement at one stage doesn't prove improvement at the next. Click each stage to see what can break there and which tool can actually see it.
1 · Discovery
The page is found and fetched by the agent that matters for that surface: crawlable links, accurate sitemaps, IndexNow for Bing, and no firewall rule quietly blocking a search bot.
What breaks it
Blocking OAI-SearchBot, Claude-SearchBot or PerplexityBot (or their published IP ranges) while trying to block training.
How to see it
Server logs by user agent, Google and Bing crawl stats, Cloudflare bot analytics.
New in 2026
Cloudflare and OpenAI's pilot surfaced hundreds of millions of previously unknown pages for indexing.
2 · Retrieval
Your page gets pulled in when an AI system runs a grounding query or fans one prompt out into several searches.
What breaks it
Thin pages, buried facts, and (per Google) manufacturing a page for every fan-out variation instead of one genuinely useful one.
How to see it
Bing's sampled grounding queries and Topics Portfolio. Vendor fan-out studies for context, not rules.
Watch the stat
Ahrefs' corrected finding: 14.4% of sampled ChatGPT responses contained fan-out queries.
3 · Answer inclusion
The page grounds an AI Overviews, AI Mode or Copilot response.
What breaks it
The Search generative AI control set to exclude, or inherited from a parent property you forgot about.
How to see it
Search Console's generative AI performance report: impressions by page, country, date and device, plus the multimodal filter.
Limits
No prompts, clicks or revenue in that report, and it may not appear when impressions are too low.
4 · Citation or mention
Your URL is cited, or your brand is named, inside the answer.
What breaks it
Treating Citation Share as traffic share, or comparing vendor share-of-voice numbers built on different prompt sets.
How to see it
Bing citations and Citation Share. Ahrefs, Semrush and Adobe trackers, with the denominator disclosed.
Remember
Microsoft calls these numbers observational and directional. No tool has Google's internal data.
5 · Visit
Someone actually clicks through to your site.
What breaks it
Interface changes. Google's May 6 link updates (inline links, previews, further reading) can move clicks without any change in source selection.
How to see it
GA4 referral and landing page reports, set up to separate AI referrers from ghost traffic.
My setup
6 · Business action
A lead, booking or purchase happens. This is the only stage your boss actually cares about.
What breaks it
Assuming vendor case-study lifts apply to you. Company claims about conversions are not independently verified.
How to see it
Conversions, CRM attribution and, for commerce, accurate merchant feeds and catalogs.
New routes
Google's UCP and Business Agent, and OpenAI's product feeds via ACP, send discovery through catalogs, not just pages.
Google Search Console
Generative AI performance (Search & Discover)
- Covers
- AI Overviews and AI Mode. Excludes Search Labs experiments.
- Shows
- Impressions by page, country, date and device. Multimodal filter since Sep 24.
- Doesn't show
- Prompts, clicks, conversions or revenue. May not appear with too few impressions.
Bing Webmaster Tools
AI Performance (public preview since Feb 10)
- Covers
- Copilot, Bing AI summaries and selected partners.
- Shows
- Citations, cited pages, sampled grounding queries, intents, Citation Share, Topics Portfolio.
- Doesn't show
- Exact prompts, competitor domains, traffic share or a quality score. Numbers are directional.
Vendor trackers
Ahrefs Brand Radar, Semrush AI Visibility Index, Adobe Brand Visibility
- Covers
- Whatever prompt set, platforms, market and period the vendor picked.
- Shows
- Modeled mentions, citations and share of voice.
- Watch for
- The denominator. Modeled impressions are not a census, and no tool has Google's internal data.
Shopping for a tracker? My AI SEO tools comparison sorts them by the job you need done, and my Claude + DataForSEO prompts include an AI citation tracking workflow you can run yourself.
AI crawler cheat sheet: GPTBot, ClaudeBot, OAI-SearchBot and PerplexityBot
The biggest technical mistake I still see is one robots.txt rule meant to "block AI" that quietly kills search visibility too. Each company now documents separate agents for separate jobs.
| Company | Search / answers | Model training | User-directed fetch |
|---|---|---|---|
| OpenAI | OAI-SearchBotPowers ChatGPT search. OpenAI recommends allowing it plus its published IP ranges. | GPTBotPotential training use. Independent of the search setting. | ChatGPT-UserSome user-directed visits. Search opt-outs may still show as navigational links. |
| Anthropic | Claude-SearchBotSearch. Disabling it can reduce search visibility. | ClaudeBotPotential training collection. | Claude-UserUser-directed access. Anthropic documents respect for robots directives. |
| Perplexity | PerplexityBotSurfaces and links sites. Not a foundation-model training crawler. | No training crawler documented in the retrieved docs | Perplexity-UserGenerally ignores robots.txt because the request is user initiated. |
| Search gen AI controlSearch Console setting for AI Overviews and AI Mode inclusion. Default: included. | Separate from the Search AI control, which does not control AI training. | Chrome agentic auditLighthouse checks llms.txt as optional for agent browsing. |
Build your AI crawler rules
Flip each agent on or off. The robots.txt snippet and warnings update live.
# Example policy: stay in AI search, opt out of training User-agent: OAI-SearchBot Allow: / User-agent: GPTBot Disallow: / User-agent: Claude-SearchBot Allow: / User-agent: ClaudeBot Disallow: / User-agent: PerplexityBot Allow: /
Google's three dials (updated Oct 21)
Whole-search exclusion
Robots metadata such as noindex, or X-Robots-Tag headers. Removes you from Search entirely.
Snippet restrictions
nosnippet, max-snippet and data-nosnippet on specific page sections.
Search generative AI control
Site-wide include, exclude or inherit for AI Overviews and AI Mode, set in Search Console. Doesn't affect regular ranking or training.
Does llms.txt do anything? It depends who's reading it
llms.txt is the most over-sold GEO tactic of the year. Here's what each source actually says, side by side. I ran my own tests on this in Markdown for AI Search: llms.txt Best Practices.
Ahrefs, May 2026: ~38,000 valid llms.txt files across 137,210 domains. Sample skews technical and SEO-aware.
No effectNeither improves nor damages visibility or rankings. Fine to keep for other services. Jun 15
OptionalAgentic audit flags server errors; a 404 is "not applicable." May 5
Rarely fetched97% of files saw no requests. Measures a root index, not page-level Markdown. Jun 15
Easier to buildAuto-builds and refreshes a store guide weekly. Doesn't prove anything reads it. Mar 31
Different toolMarkdown for Agents serves a page as Markdown on Accept: text/markdown. Not an llms.txt index. Feb 12
11 GEO claims to retire or qualify
If any of these are in your decks, client reports or old blog posts, it's time for an update. Think you know the real answer? Tap each one to bust it.
Google has not acknowledged GEO or AEO.
Outdated. The May 2026 guide addresses both terms directly.See item 11
Google provides no separate AI visibility reporting.
Outdated. Dedicated reporting rolled out worldwide by August 31.See item 5
Google's AI reporting reveals all prompts, clicks and revenue.
Unsupported. Report on the dimensions and metrics it actually documents.See item 5
You must allow a training crawler to appear in AI search.
Confuses separate permissions. Search and training agents are independent.Try the crawler builder
An llms.txt file guarantees citations.
Unsupported. Separate platform statements, delivery features and experiments.See the llms.txt check
All AI platforms require short chunks or FAQ schema.
Not a universal rule. Google rejects mandatory chunks; Microsoft's structure advice is Microsoft's.See items 11 and 21
A citation is a click, a recommendation or a sale.
Different events with different denominators.See the output chain
Bing's report represents the whole AI market.
It covers Microsoft surfaces and selected partners only.See item 19
The Cloudflare and OpenAI pilot proves more revenue.
Its early finding is about discovery and indexing. Outcome work is still in progress.See item 22
Ahrefs found 85.6% of ChatGPT responses never search.
Overstates the corrected result: 14.4% contained fan-out queries.See item 33
An SEO vendor's official post is an official Google rule.
Attribute it to the vendor and keep its evidence type. Google says third-party tools can't guarantee results.See item 8
Related: chasing "inauthentic mentions" is exactly the kind of manipulation both Google and Bing called out this year. I break down how those schemes work in Black Hat SEO Techniques: 6 Ways Sites Fake Importance.
10 changes to make to your GEO playbook
My editorial recommendations based on the research above, not extra platform announcements. Check them off as you go; your progress saves in this browser. For the step-by-step version, use my 16-step generative engine optimization checklist.
Generative engine optimization questions, answered
What is generative engine optimization (GEO)?
GEO is the work of making your content discoverable, retrievable and citable in AI-generated answers such as Google AI Overviews and AI Mode, Microsoft Copilot, ChatGPT search and Perplexity. Google's May 2026 guide treats it as part of SEO, while Microsoft frames it around grounding: the link between AI answers and current, authoritative sources.
Is GEO replacing SEO?
Not according to the platforms. Google's May 15, 2026 guide places GEO and AEO inside SEO and puts original, useful content and ordinary technical eligibility first. Bing's guidelines tie grounding eligibility to discovery, indexing, sitemaps and crawlable links. Classic SEO is the foundation GEO runs on.
What is the difference between AEO and GEO?
They overlap heavily. AEO (answer engine optimization) and GEO (generative engine optimization) both describe earning visibility inside AI answers. Google's guide addresses both terms together as part of SEO. Microsoft uses GEO specifically and measures it through citations in Bing Webmaster Tools.
Does llms.txt help Google rankings?
No. Google's June 15, 2026 changelog says llms.txt files neither improve nor damage Google Search visibility or rankings. Chrome Lighthouse treats the file as optional, and an Ahrefs log study found 97% of about 38,000 files received no requests in May 2026. It may still matter for other systems if they actually fetch it, so test before you count on it.
Do I have to allow GPTBot to appear in ChatGPT search?
No. OpenAI documents OAI-SearchBot for ChatGPT search and GPTBot for potential training use, and says the two preferences are independent. You can allow OAI-SearchBot and block GPTBot.
What is ClaudeBot, and how is it different from Claude-SearchBot?
Anthropic documents three agents: ClaudeBot for potential training collection, Claude-SearchBot for search, and Claude-User for user-directed fetches. Blocking ClaudeBot is a training decision; blocking Claude-SearchBot or Claude-User can reduce search visibility or retrieval.
Can I see AI Overviews and AI Mode performance in Search Console?
Yes. Google's generative AI performance reports rolled out worldwide on August 31, 2026. The Search report covers AI Overviews and AI Mode with impressions by page, country, date and device. It is not a prompt, click, conversion or revenue dashboard, so pair it with analytics for outcomes.
How do I keep my site out of AI Overviews without leaving Google Search?
Use the Search generative AI control in Search Console. It manages whether your links and content can appear in and ground AI Overviews and AI Mode. Inclusion is the default for top-level properties, child properties can inherit, and it does not affect ranking elsewhere in Search or AI training.
More GEO research from Optimization Theory
Want to know where you stand in AI search?
I'll check your crawler access, Search Console and Bing AI data, and show you exactly where you're cited, where you're missing, and what to fix first.
About the author
Jourdan Rombough is the founder of Optimization Theory and has spent 15+ years in technical SEO, enterprise search and, lately, AI visibility. He tests GEO tactics on his own sites before recommending them to anyone else. Connect on LinkedIn.
Method and research limits
- Research cutoff: October 22, 2026, America/New_York. This is a dated synthesis of public evidence, not a continuous monitor, and it doesn't claim to capture every social post, conference remark or podcast.
- Broad AI model launches, ordinary ad updates and unrelated SEO interface changes were left out unless they clarified discovery, source access, visibility or measurement.
- Documentation can change without preserving its original wording. Dates identify the announcement or displayed update, not a full history of every sentence.
- The Bing Webmaster Guidelines help app returned a JavaScript shell; the guidance was read from the search index's text for the official URL. Undated OpenAI and Perplexity crawler pages are labeled as such.
- One specific Perplexity Premium Sources article couldn't be retrieved; item 27 uses an accessible official announcement and the official blog index.
- No website settings, crawler permissions or external accounts were changed for this research.
More Writing & Research by Jourdan Rombough
View all research
If AI can't find you, understand you and trust you, it won't recommend you.
Keywords tell you what they typed. Intent tells you what they meant. Fan-out tells you what they'll ask next.
Be a thing, not a string. AI remembers entities, not keywords.
The click isn't dead. It just has to be earned twice.
Schema is the business card you hand to every machine that reads your site.
If Googlebot can't reach it and AI can't parse it, it doesn't exist.
Good SEO grows quietly. Then all at once.
