Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.brightdata.com/llms.txt

Use this file to discover all available pages before exploring further.

The AI Scraper Studio API is a set of endpoints that let you create Scraper Studio scrapers with AI and update existing scrapers using Self-Healing tool, without using the UI. This page explains which endpoint to call and in what order. Both flows are asynchronous: you start an AI job, then check its progress until it completes.

​
Workflow 1: Create a new scraper with AI

Use these endpoints when you want AI to generate a scraper (schema + code) from scratch for a target site.
  1. Create the scraper entity
    → Create scraper template
  2. Start the AI generation job (schema + code)
    → Trigger AI Flow to create code
  3. Poll until the AI job finishes and returns the result
    → AI job progress

​
Workflow 2: Update an existing scraper with Self-Healing

Use these endpoints when you already have a scraper and want to fix or modify it using a prompt.
  1. Start a Self-Healing refactor job
    → Trigger Self-Healing
  2. Poll until the refactor job finishes or pauses for user input
    → Self-Healing job progress
  3. If progress returns status: "pending_answer", approve or reject the proposed diff
    → Resume Self-Healing job
  4. After the scraper is ready, initiate collection
    → Initiate collection
Once created or updated, run it using the standard Scraper Studio API initiation endpoints (manual trigger, schedule, queue, etc.).
Not sure which to use?
New to the API?See: Getting started with the API