Extract API

Endpoint

POST https://api.serpex.dev/api/crawl

Authentication

Authorization: Bearer sk_...

Request Parameters

What Gets Extracted

Extraction handles text-based content. Binary documents are not extracted.

Supported: HTML and XHTML, plain text, Markdown, CSV, JSON (including +json types such as application/ld+json), and XML — which covers RSS and Atom feeds and sitemap.xml.

Not supported: PDFs, images (including SVG), video, audio, archives, office documents, fonts and other binaries.

Rejections are per URL, not per request — the remaining URLs in the same call are still extracted normally — and a rejected URL is never charged. A URL is rejected on its file extension before any network request is made where possible, and otherwise on the response's content-type and size. A binary served under a text content-type is still detected and rejected.

We also honour each site's robots.txt. A URL disallowed by the target site returns error_type: "robots_disallowed" and is not charged.

Code Examples

curl -X POST https://api.serpex.dev/api/crawl \
  -H "Authorization: Bearer sk_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com", "https://httpbin.org"],
    "format": "markdown"
  }'

import { SerpexClient } from "serpex";

const client = new SerpexClient("sk_your_api_key");

// Markdown output (default) — ready for LLM processing
const result = await client.extract({
  urls: ["https://example.com", "https://httpbin.org"],
});

console.log(result.results[0].markdown);
console.log(`Credits used: ${result.metadata.credits_used}`);

// HTML output — useful for structured data extraction
const htmlResult = await client.extract({
  urls: ["https://example.com"],
  format: "html",
});

console.log(htmlResult.results[0].html);

from serpex import SerpexClient

client = SerpexClient("sk_your_api_key")

# Markdown output (default) — ready for LLM processing
result = client.extract({
    "urls": ["https://example.com", "https://httpbin.org"],
})

print(result.results[0].markdown)
print(f"Credits used: {result.metadata.credits_used}")

# HTML output — useful for structured data extraction
html_result = client.extract({
    "urls": ["https://example.com"],
    "format": "html",
})

print(html_result.results[0].html)

Markdown vs HTML Output

Markdown (format: "markdown") strips navigation, ads, and boilerplate — leaving only the main content in a clean format that LLMs can read directly.

HTML (format: "html") returns the full rendered HTML of the page. Use this when you need to parse structured data with CSS selectors or XPath.

Response

{
  "results": [
    {
      "url": "https://example.com",
      "success": true,
      "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
      "status_code": 200
    },
    {
      "url": "https://example.com/missing",
      "success": false,
      "markdown": "",
      "status_code": 404,
      "error": "HTTP 404: Not Found",
      "error_type": "http"
    }
  ],
  "metadata": {
    "total_urls": 2,
    "processed_urls": 2,
    "successful_crawls": 1,
    "failed_crawls": 1,
    "credits_used": 1,
    "cached_free": 0,
    "response_time": 1850,
    "timestamp": "2025-06-21T10:30:00.000Z"
  }
}

Response Fields

Credits

• 1 credit per successfully extracted URL (normal mode).
• 5 credits per successfully extracted URL with stealth: true.
• 0 credits for any URL served from cache.
• Failed extractions are not charged.