Skip to main content

ProxyConfiguration

Configures a connection to a proxy server with the provided options.

Proxy servers are used to prevent target websites from blocking your crawlers based on IP address rate limits or blacklists. The default servers used by this class are managed by Apify Proxy. To be able to use Apify Proxy, you need an Apify account and access to the selected proxies. If you provide no configuration option, the proxies will be managed automatically using a smart algorithm.

If you want to use your own proxies, use the proxy_urls or new_url_function constructor options. Your list of proxy URLs will be rotated by the configuration, if this option is provided.

Index

Methods

Methods

__init__

  • __init__(*, password, groups, country_code, proxy_urls, new_url_function, tiered_proxy_urls, _actor_config, _apify_client): None

  • Create a ProxyConfiguration instance.

    It is highly recommended to use Actor.create_proxy_configuration() instead of this.

    Parameters

    • optionalkeyword-onlypassword: str | None = None

      Password for the Apify Proxy. If not provided, will use os.environ['APIFY_PROXY_PASSWORD'], if available.

    • optionalkeyword-onlygroups: list[str] | None = None

      Proxy groups which the Apify Proxy should use, if provided.

    • optionalkeyword-onlycountry_code: str | None = None

      Country which the Apify Proxy should use, if provided.

    • optionalkeyword-onlyproxy_urls: list[str | None] | None = None

      Custom proxy server URLs which should be rotated through.

    • optionalkeyword-onlynew_url_function: _NewUrlFunction | None = None

      Function which returns a custom proxy URL to be used.

    • optionalkeyword-onlytiered_proxy_urls: list[list[str | None]] | None = None

      Proxy URLs arranged into tiers

    • optionalkeyword-only_actor_config: Configuration | None = None
    • optionalkeyword-only_apify_client: ApifyClientAsync | None = None

    Returns None

initialize

  • async initialize(): None

  • Check if using proxy, if so, check the access.

    Load the Apify Proxy password from API (only if not passed to constructor or through env var).

    Only called if Apify Proxy configuration is used. Also checks if country has access to Apify Proxy groups if the country code is provided.

    You should use the Actor.create_proxy_configuration function to create a pre-initialized ProxyConfiguration instance instead of calling this manually.

    Returns None

new_proxy_info

  • async new_proxy_info(session_id, request, proxy_tier): ProxyInfo | None

  • Create a new ProxyInfo object.

    Use it if you want to work with a rich representation of a proxy URL. If you need the URL string only, use ProxyConfiguration.new_url.

    Parameters

    • optionalsession_id: str | None = None

      Represents the identifier of a proxy session (https://docs.apify.com/proxy#sessions). All the HTTP requests going through the proxy with the same session identifier will use the same target proxy server (i.e. the same IP address). The identifier must not be longer than 50 characters and include only the following: 0-9, a-z, A-Z, ".", "_" and "~".

    • optionalrequest: Request | None = None

      request for which the proxy info is being issued, used in proxy tier handling.

    • optionalproxy_tier: int | None = None

      allows forcing the proxy tier to be used.

    Returns ProxyInfo | None

Page Options