www.firecrawl.dev.

level 4: agent integration

Firecrawl is a web scraping and crawling API service designed for AI applications, enabling developers to extract clean, structured data from any website. It converts web pages into LLM-ready markdown or structured data, handling JavaScript rendering, rate limiting, and anti-bot measures automatically. Firecrawl supports full site crawling, single page scraping, and search capabilities, making it ideal for building AI agents and RAG pipelines that need reliable web data.

LEVEL 1 / BASIC ACCESSIBILITY — 4/6

Uses semantic HTML elements (header, nav, main, article, section, footer) instead of just divs FAIL

Has proper meta tags (title, description, og:tags, twitter:card) PASS

Includes Schema.org JSON-LD structured data FAIL

Does not block automated access with CAPTCHAs on public content PASS

Content is server-side rendered (visible in HTML source, not just JS-rendered) PASS

Uses clean, readable URLs (no excessive query params or hash fragments) PASS

LEVEL 2 / DISCOVERABILITY — 5/6

Has a robots.txt that allows legitimate bot access PASS

Provides an XML sitemap PASS

Has a /llms.txt file describing the site for LLMs PASS

Publishes an OpenAPI/Swagger specification for its API FAIL

Has comprehensive, machine-readable documentation PASS

Primary content is text-based (not locked in images/videos/PDFs) PASS

LEVEL 3 / STRUCTURED INTERACTION — 4/6

Provides a structured REST or GraphQL API PASS

API returns JSON responses with consistent schema PASS

API supports search and filtering parameters PASS

Has an A2A agent card at /.well-known/agent.json FAIL

Rate limits are documented and return proper 429 responses with Retry-After FAIL

API returns structured error responses with error codes and messages PASS

LEVEL 4 / AGENT INTEGRATION — 4/6

Provides an MCP (Model Context Protocol) server PASS

Supports WebMCP for browser-based agent interaction FAIL

API supports write operations (POST/PUT/PATCH/DELETE), not just reads PASS

Supports agent-friendly authentication (API keys, OAuth client credentials) PASS

Supports webhooks for event notifications PASS

Write operations support idempotency keys FAIL

LEVEL 5 / AUTONOMOUS OPERATION — 2/6

Supports event streaming (SSE, WebSockets) for real-time updates PASS

Supports agent-to-agent capability negotiation FAIL

Has a subscription/management API for agents FAIL

Supports multi-step workflow orchestration PASS

Can proactively notify agents of relevant changes FAIL

Supports cross-service handoff between agents FAIL