cuery.seo.aio.scrapingdog#

API helpers to access Google AI Overview in Google SERP results via ScrapingDog.

Scraping Dog#

Implements a convenience function query_google() that mirrors the shape of cuery.search provider helpers by returning a SearchResult (answer + sources) extracted from Google AI Overview (aka AI Overviews / AI Summary) when available.

The ScrapingDog API exposes (at least) two relevant endpoints:

https://api.scrapingdog.com/google – standard SERP results. In some cases the AI Overview content may be embedded directly in the JSON payload (future proofing – not currently documented in retrieved snippets but handled here).
https://api.scrapingdog.com/google/ai_overview – dedicated endpoint for AI Overview content when Google requires a secondary fetch.

Functions#

`flatten_text_blocks`(blocks)	Convert list of `text_blocks` to a single answer string.
`parse_aio`(aio)	Extract AI Overview into a `SearchResult`.
`aio_api_url`(aio)	Extract the API URL from the aio dict, if available.
`query`(prompt[, country, language, validate, log, session])	Execute a Google search via ScrapingDog and extract AI Overview.
`gather`(prompts[, country, language, validate, log, ...])	Create zero-argument coroutine factories (with policies) for many prompts.

Module Contents#

cuery.seo.aio.scrapingdog.flatten_text_blocks(blocks)#

Convert list of text_blocks to a single answer string.

Supported block types (based on docs sample): paragraph and list. A list block contains a list key with items each having snippet. Unknown types are ignored (future proof).

Parameters:: blocks (collections.abc.Iterable[dict[str, Any]] | None)
Return type:: str

cuery.seo.aio.scrapingdog.parse_aio(aio)#

Extract AI Overview into a SearchResult.

Expected structure (subset): {

“ai_overview”: {
“text_blocks”: […], “references”: [ {“title”: str, “link”: str, …}, … ]

}

}

Return type:: cuery.search.SearchResult

cuery.seo.aio.scrapingdog.aio_api_url(aio)#

Extract the API URL from the aio dict, if available.

Return type:: str | None

async cuery.seo.aio.scrapingdog.query(prompt, country=None, language=None, validate=True, log=False, session=None)#

Execute a Google search via ScrapingDog and extract AI Overview.

Parameters:

prompt (str)
country (str | None)
language (str | None)
validate (bool)
log (bool)
session (aiohttp.ClientSession | None)

Return type:

cuery.search.SearchResult | dict[str, Any]

async cuery.seo.aio.scrapingdog.gather(prompts, country=None, language=None, validate=True, log=False, session=None, policies=None, execute=True)#

Create zero-argument coroutine factories (with policies) for many prompts.