CRCrawl CensusAsk before you fetch: will this domain serve your crawler, refuse it, or charge it?Browser & webRemoteMCP Registry
CRcrawl readinessCheck what AI crawlers may read and what a page permits, generate llms.txt and schema.Browser & webLocalMCP Registry
CRCrawl4Agentcrawl4ai-compatible web crawler API: POST a URL, get clean LLM-ready Markdown with fit_markdown...Browser & webRemoteMCP Registry