API & MCP

Everything a person can do on Structured Page Extractor, a program or AI agent can do with the same inputs and get the identical result.

HTTP API

Structured Page Extractor

Reads one public HTML page (or HTML you paste) with a small recipe of CSS selectors, text/attribute/regex rules and an optional same-site next-page link, and returns a table as JSON and CSV. Can open each row’s link on the same site (up to 20 detail pages) to read more fields there. Compares with an earlier run to show new, removed and changed rows, and suggests replacement selectors when a site changed. robots.txt is checked first; no JavaScript, cookies or logins.

Inputs of Structured Page Extractor
FieldTypeDefaultDescription
urltext—Public page address. http(s) only. robots.txt is checked first; pages behind logins, paywalls or bot checks are not read.
htmltextarea—Or paste the page HTML. Used instead of the address; nothing is fetched.
baseUrltext—Address of the pasted page (for relative links)
recipetextarea"{\n \"mode\": \"list\",\n \"item\": \"li.notice\",\n \"fields\": [\n {\n Recipe (JSON). JSON: {"mode":"list","item":"li.notice","fields":[{"name":"title","selector":"a"},{"name":"link","selector":"a::attr(href)","type":"url"},{"name":"price","selector":".price","type":"number"}],"next":"a[rel=next]","maxPages":2,"key":"link","detail":{"link":"link","fields":[{"name":"deadline","selector":".deadline"}],"max":10}}. Field options: take text|own-text|attr (+ attr), regex (first group kept), type text|number|url, all (join every match). Optional detail: open each row’s link (same site, robots.txt checked, up to 20 pages) and read more fields there.
previoustextarea—Earlier run to compare (JSON export or CSV, optional)
returnHtmlbooleanfalseInclude the first page’s HTML in the result (up to 1 MB, for a live preview)

Example request

Notices list from pasted HTML

curl -X POST https://obscuriqpageextractor.usefulweb.site/api/v1/calculate -H 'content-type: application/json' -d '{"html":"<!doctype html><html lang=\"en\"><head><title>Northwind Notices — public register</title></head><body>\n<main><h1>Public notices</h1>\n<ul class=\"notices\">\n<li class=\"notice\"><a class=\"title\" href=\"/notices/101\">Road works on Harbour Street</a><span class=\"date\">2026-11-02</span><span class=\"fee\">€ 1.250,00</span></li>\n<li class=\"notice\"><a class=\"title\" href=\"/notices/102\">New opening hours: library</a><span class=\"date\">2026-11-05</span><span class=\"fee\">free</span></li>\n<li class=\"notice featured\"><a class=\"title\" href=\"/notices/103\">Tender: school roof repair</a><span class=\"date\">2026-11-09</span><span class=\"fee\">€ 84.000</span></li>\n</ul>\n<nav class=\"pager\"><a rel=\"next\" href=\"/notices?page=2\">Next</a></nav></main></body></html>","baseUrl":"https://notices.example/notices","recipe":"{\"mode\":\"list\",\"item\":\"li.notice\",\"fields\":[{\"name\":\"title\",\"selector\":\"a.title\",\"take\":\"text\",\"type\":\"text\"},{\"name\":\"link\",\"selector\":\"a.title\",\"take\":\"attr\",\"attr\":\"href\",\"type\":\"url\"},{\"name\":\"date\",\"selector\":\".date\",\"take\":\"text\",\"type\":\"text\",\"regex\":\"(\\\\d{4}-\\\\d{2}-\\\\d{2})\"},{\"name\":\"fee (EUR)\",\"selector\":\".fee\",\"take\":\"text\",\"type\":\"number\"}],\"next\":\"a[rel=next]\",\"maxPages\":1,\"key\":\"link\"}"}'

Every result contains

inputs (normalised, defaults applied), summary, result (validated against the published schema), units, assumptions, sources, warnings, computedAt and links.

MCP (Model Context Protocol)

{
  "mcpServers": {
    "obscuriq-page-extractor": { "type": "http", "url": "https://obscuriqpageextractor.usefulweb.site/mcp" }
  }
}