ApifyCacheStorage

A Scrapy cache storage that uses the Apify KeyValueStore to store responses.

It can be set as a storage for Scrapy's built-in HttpCacheMiddleware, which caches responses to requests. See HTTPCache middleware settings (prefixed with HTTPCACHE_) in the Scrapy documentation for more information. Requires the asyncio Twisted reactor to be installed.

Index

Methods

• __init__
• close_spider
• open_spider
• retrieve_response
• store_response

Methods

__init__

• __init__(settings): None

• Parameters

  • settings: BaseSettings

  Returns None

close_spider

• close_spider(_, current_time): None

• Close the cache storage for a spider.

  ---

  Parameters

  • _: Spider

  • optionalcurrent_time: int | None = None

  Returns None

open_spider

• open_spider(spider): None

• Open the cache storage for a spider.

  ---

  Parameters

  • spider: Spider

  Returns None

retrieve_response

• retrieve_response(_, request, current_time): Response | None

• Retrieve a response from the cache storage.

  ---

  Parameters

  • _: Spider

  • request: Request

  • optionalcurrent_time: int | None = None

  Returns Response | None

store_response

• store_response(_, request, response): None

• Store a response in the cache storage.

  ---

  Parameters

  • _: Spider

  • request: Request

  • response: Response

  Returns None