ANSWERS

What is Google-Extended and should I allow it?

Google-Extended is a robots.txt token for controlling use of your content in training for Gemini and Vertex AI generative models, and in grounding in Gemini Apps and Grounding with Google Search on Vertex AI. It has no separate HTTP user-agent. It does not affect Google Search inclusion or ranking, including AI Overviews, because Googlebot Search access is separate. ye5's GEO Web audit checks whether your machine-access setup reflects that choice on purpose.

What Google-Extended controls

Google-Extended is a robots.txt user-agent token Google publishes for controlling use of your content in training for Gemini and Vertex AI generative models, and in grounding in Gemini Apps and Grounding with Google Search on Vertex AI. It has no separate HTTP user-agent of its own. Omitting the token leaves the default in place, and broader robots rules that could apply still have to be checked. It sits apart from the crawlers Google uses to index and retrieve your pages.

How Google-Extended differs from Googlebot

Googlebot indexes your site for Search and for any Google surface built on that index, including AI Overviews. Google-Extended does not affect Google Search inclusion or ranking, including AI Overviews, and Googlebot Search access is set separately. The token governs whether Google can use your content for training and for the named grounding uses. The two are set independently in robots.txt (see: What should go in robots.txt for AI crawlers?). Training directives and the separate search and user-fetch agents are different controls (see: Does blocking AI training bots hurt my visibility in AI search?).

Should you allow it or block it?

Blocking it restricts the specified Gemini and Vertex AI grounding uses as well as training, while Google Search visibility is governed separately. No confirmed link ties the token to citation frequency, so allowing it is not a guaranteed citation strategy. Owners who don't want their published work used for training or those grounding uses have a legitimate reason to block it. Owners with no such preference can leave the default in place.

ye5's GEO Web audit checks machine access as part of a full-site review — whether Google-Extended, GPTBot, and the rest are blocked on purpose or by accident inside a blanket rule.

How to set the directive

Setting the directive takes a two-line robots group: a User-agent line and a Disallow line in a file you already control.

—Write User-agent: Google-Extended on one line, then Disallow: / on the next, to opt out of training and the named grounding uses.
—Omitting the token leaves the default in place, and broader rules that could apply must be checked.
—Keep that group separate from any blanket User-agent: * block — a wildcard Disallow there can catch Googlebot too, which does affect Search.
—Re-fetch your own robots.txt after you publish the change to confirm it saved the way you intended.

A two-line robots group is easy to get right and easy to overlook inside a longer file — that's the kind of gap a machine-access review is built to catch.

FAQ

Does blocking Google-Extended remove my site from Google Search?

No. Google-Extended is an opt-out covering training for Gemini and Vertex AI generative models and grounding in Gemini Apps and Grounding with Google Search on Vertex AI. It does not affect Search inclusion or ranking, including AI Overviews, and Googlebot crawls for Search under its own user-agent line.

Does allowing Google-Extended get my brand cited more by Gemini?

No confirmed link exists between the token and citation frequency. Blocking it also affects grounding in Gemini Apps and Grounding with Google Search on Vertex AI, so the choice is about those uses and training, not a citation strategy.

Is Google-Extended the only AI training token I need to know?

No. GPTBot and ClaudeBot are separate opt-outs for other providers, each set with its own robots group under its own user-agent name.