Obscura Guide

Obscura Scraping Guide

Use Claude Code & Codex to scrape websites – no coding experience needed.

What This Does

This guide walks you through building a simple web scraper using three tools: Obscura (a headless browser), Claude Code (Anthropic's AI coding agent), and Codex (OpenAI's AI coding agent). The AI agents write and fix all the code for you – you just run commands. By the end, your scraper visits websites, waits for content to load, and saves extracted data as JSON and CSV files.

How It Works

Three layers work together

Claude Code / Codex → writes & debugs your code

Your scraper (Node.js) → controls the browser

Obscura → renders the website like a real browser Target website → the data you want The AI agents don't scrape directly. They write local code that controls Obscura, which acts as the browser.

What You Need

–

Terminal (built into Mac/Linux; use PowerShell on Windows)

–

Git - https://git-scm.com

– Node.js 20+ – https://nodejs.org – Obscura – https://github.com/h4ckf0r0day/obscura/releases/latest –

Claude Code - https://claude.ai/code

–

Codex CLI - installed via npm after Node.js is set up

Install Obscura

macOS (Apple Silicon / M1-M4)

curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-aarch64-macos.tar.gz

tar xzf obscura-aarch64-macos.tar.gz

chmod +x obscura obscura-worker

xattr -dr com.apple.quarantine ./obscura ./obscura-worker

macOS (Intel)

curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-macos.tar.gz

tar xzf obscura-x86_64-macos.tar.gz

chmod +x obscura obscura-worker

xattr -dr com.apple.quarantine ./obscura ./obscura-worker

Linux

curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-linux.tar.gz

tar xzf obscura-x86_64-linux.tar.gz

chmod +x obscura obscura-worker

Windows

Download the .zip from the releases page, extract it, open PowerShell in that folder, then run: .\obscura.exe fetch https://example.com –eval "document.title"

Quick test to confirm Obscura works

./obscura fetch https://example.com –eval "document.title"

Create the Project

mkdir obscura-scraper
cd obscura-scraper
git init
npm init -y
npm pkg set type=module
npm i playwright-core
mkdir -p src data output bin

Copy the Obscura binary into the project

cp /path/to/obscura ./bin/obscura cp /path/to/obscura-worker ./bin/obscura-worker

chmod +x ./bin/obscura ./bin/obscura-worker

Create data/urls.txt and add the URLs to scrape, one per line

https://quotes.toscrape.com/ https://quotes.toscrape.com/page/2/

Install Claude Code & Codex

Claude Code
curl -fsSL https://claude.ai/install.sh | bash
# or with Homebrew:
brew install --cask claude-code
Codex CLI
npm i -g @openai/codex

Let Claude Code Build the Scraper

Start Claude Code inside your project folder, then paste this prompt

cd obscura-scraper
claude

Build a scraper in this project.

• Connect to Obscura at ws://127.0.0.1:9222 using playwright-core
• Read URLs from data/urls.txt
• Add a 1.5 second delay between pages
• Extract quote text, author, and tags from quotes.toscrape.com
• Save results to output/quotes.json and output/quotes.csv
• Add npm scripts: "npm run obscura" and "npm run scrape"

Claude will create src/scrape.js and update package.json. Review and approve each command it suggests.

Run the Scraper

Open two terminal windows at the same time.

Terminal 1 – Start Obscura

npm run obscura

Keep this running. Obscura is now acting as your headless browser.

Terminal 2 – Run the Scraper

npm run scrape

Output files are saved to output/quotes.json and output/quotes.csv.

Improve with Codex

Start Codex and paste this prompt to review and harden the scraper

codex

Review src/scrape.js and improve it

1. Add better logging
2. Move selectors to config/selectors.json
3. Add "npm test" with basic tests
4. Show a diff before making any changes

Adapting to Your Own Website

Once the test scraper works, use this prompt in Claude Code or Codex

Adapt the scraper for this target

URL: <your website URL>

Fields to extract

• <field 1>
• <field 2>

Put selectors in config/selectors.json. Run a 2-page smoke test first.

Common Issues

Permission denied → chmod +x ./bin/obscura ./bin/obscura-worker

macOS says binary is damaged

→ xattr -dr com.apple.quarantine ./bin/obscura ./bin/obscura-worker Port 9222 already in use → lsof -i :9222 → kill that process, or change the port

ECONNREFUSED 127.0.0.1:9222

→ Obscura isn't running. Go to Terminal 1 and run: npm run obscura Empty fields in output → Ask Claude Code to debug the CSS selectors using the saved HTML Cannot use import statement → npm pkg set type=module

Quick Reference

npm run obscura # Terminal 1 - start the browser
npm run scrape # Terminal 2 - run the scraper
claude # Start Claude Code
codex # Start Codex

Obscura: github.com/h4ckf0r0day/obscura | Claude Code: code.claude.com | Codex

developers.openai.com/codex