API & MCP
Everything a person can do on Structured Page Extractor, a program or AI agent can do with the same inputs and get the identical result.
HTTP API
POST https://obscuriqpageextractor.usefulweb.site/api/v1/calculate– run Structured Page ExtractorGET https://obscuriqpageextractor.usefulweb.site/api/v1/tool– inputs, output schema, examples- OpenAPI 3.1: /openapi.json · /openapi.yaml
- Errors:
application/problem+json;422includes per-fielderrors.
Structured Page Extractor
Reads one public HTML page (or HTML you paste) with a small recipe of CSS selectors, text/attribute/regex rules and an optional same-site next-page link, and returns a table as JSON and CSV. Can open each row’s link on the same site (up to 20 detail pages) to read more fields there. Compares with an earlier run to show new, removed and changed rows, and suggests replacement selectors when a site changed. robots.txt is checked first; no JavaScript, cookies or logins.
| Field | Type | Default | Description |
|---|---|---|---|
url | text | — | Public page address. http(s) only. robots.txt is checked first; pages behind logins, paywalls or bot checks are not read. |
html | textarea | — | Or paste the page HTML. Used instead of the address; nothing is fetched. |
baseUrl | text | — | Address of the pasted page (for relative links) |
recipe | textarea | "{\n \"mode\": \"list\",\n \"item\": \"li.notice\",\n \"fields\": [\n {\n | Recipe (JSON). JSON: {"mode":"list","item":"li.notice","fields":[{"name":"title","selector":"a"},{"name":"link","selector":"a::attr(href)","type":"url"},{"name":"price","selector":".price","type":"number"}],"next":"a[rel=next]","maxPages":2,"key":"link","detail":{"link":"link","fields":[{"name":"deadline","selector":".deadline"}],"max":10}}. Field options: take text|own-text|attr (+ attr), regex (first group kept), type text|number|url, all (join every match). Optional detail: open each row’s link (same site, robots.txt checked, up to 20 pages) and read more fields there. |
previous | textarea | — | Earlier run to compare (JSON export or CSV, optional) |
returnHtml | boolean | false | Include the first page’s HTML in the result (up to 1 MB, for a live preview) |
Example request
Notices list from pasted HTML
curl -X POST https://obscuriqpageextractor.usefulweb.site/api/v1/calculate -H 'content-type: application/json' -d '{"html":"<!doctype html><html lang=\"en\"><head><title>Northwind Notices — public register</title></head><body>\n<main><h1>Public notices</h1>\n<ul class=\"notices\">\n<li class=\"notice\"><a class=\"title\" href=\"/notices/101\">Road works on Harbour Street</a><span class=\"date\">2026-11-02</span><span class=\"fee\">€ 1.250,00</span></li>\n<li class=\"notice\"><a class=\"title\" href=\"/notices/102\">New opening hours: library</a><span class=\"date\">2026-11-05</span><span class=\"fee\">free</span></li>\n<li class=\"notice featured\"><a class=\"title\" href=\"/notices/103\">Tender: school roof repair</a><span class=\"date\">2026-11-09</span><span class=\"fee\">€ 84.000</span></li>\n</ul>\n<nav class=\"pager\"><a rel=\"next\" href=\"/notices?page=2\">Next</a></nav></main></body></html>","baseUrl":"https://notices.example/notices","recipe":"{\"mode\":\"list\",\"item\":\"li.notice\",\"fields\":[{\"name\":\"title\",\"selector\":\"a.title\",\"take\":\"text\",\"type\":\"text\"},{\"name\":\"link\",\"selector\":\"a.title\",\"take\":\"attr\",\"attr\":\"href\",\"type\":\"url\"},{\"name\":\"date\",\"selector\":\".date\",\"take\":\"text\",\"type\":\"text\",\"regex\":\"(\\\\d{4}-\\\\d{2}-\\\\d{2})\"},{\"name\":\"fee (EUR)\",\"selector\":\".fee\",\"take\":\"text\",\"type\":\"number\"}],\"next\":\"a[rel=next]\",\"maxPages\":1,\"key\":\"link\"}"}'
Every result contains
inputs (normalised, defaults applied), summary, result (validated against the published schema), units, assumptions, sources, warnings, computedAt and links.
MCP (Model Context Protocol)
- Streamable HTTP endpoint:
https://obscuriqpageextractor.usefulweb.site/mcp– protocol revisions 2026-07-28 (stateless) and 2025-11-25 / 2025-06-18 / 2025-03-26 (initialize handshake). - Tools:
obscuriq_page_extractor - Local stdio connector:
node src/mcp-stdio.ts --tool=obscuriq-page-extractor
{
"mcpServers": {
"obscuriq-page-extractor": { "type": "http", "url": "https://obscuriqpageextractor.usefulweb.site/mcp" }
}
}