Platform Settings

Global Defaults

Execution

Default limits for scraper workers, retries, and timeouts. A scraper with no explicit override inherits these values.

Parallel workers for new scraper runs. Leave blank for no platform default (scraper class default applies).

Retry attempts before failing the run. Leave blank for no platform default.

Per-request network timeout. Leave blank for no platform default.

seconds

Delay inserted between outbound requests. Leave blank for no platform default.

seconds

Grace period before the worker is asked to stop.

seconds

Hard wall-clock cap for the run.

seconds

Stops runs that stop making progress. Leave blank to disable.

seconds

Default verbosity for scraper logs.

Response cache TTL for all scrapers without an explicit override.

seconds

Proxy Routing

Default proxy behaviour and pool validation parameters. Individual pools and scrapers can override these.

Consecutive failures before a proxy is marked unhealthy during a run.

failures

Number of retries when validating a proxy during preflight (0 = test once).

Default URL used to validate proxies when no pool-specific URL is set. Bootstrapped from PROXY_TEST_URL env var on first load.

Default validation timeout for pools that have no explicit timeout set.

seconds

Default validation frequency for pools.

minutes

Default threshold below which a pool triggers an immediate refresh.

proxies

Default staleness threshold for pre-run pool refresh. Leave blank to use each pool's refresh interval.

minutes

Reservoir

Defaults for the active proxy reservoir used during scraper runs.

How often to re-test proxies in the active reservoir mid-run.

minutes

API

Manage the REST API interface for external orchestration.

Enable the REST API endpoints and automatic Swagger documentation.

If enabled, all API requests (except docs) must provide a valid API key or session.

The HTTP header name to check for the API key.

API authentication uses managed API key records only. Configure keys from the API Access manager. Open Swagger API docs.

Manage API keys

Worker Version Policy

Controls worker eligibility and rollout target behavior.

Worker Nodes

WorkerStatusDrainingVersionQueues
worker-prod-1 online no — worker.worker-prod-1, celery, scrapers.api, scrapers.browser, management, proxy-fetch, proxy-validate

Delivery Targets

Lifecycle-triggered delivery and notification endpoints. Add targets here then attach hooks per-scraper in scraper settings.

No delivery targets configured yet.
+ Add Target