What big tech infrastructure is
Infrastructure companies that sit between websites and AI crawlers can charge AI companies for access to content. They monetize public web content rather than internal company records, so they sit in their own category.
How it pays
The infrastructure company charges the AI company when it accesses content and pays the site owner. The price, the payout schedule and any minimum payout are set in the program's terms.
Companies compared
Every value below is as published or reported, with its source. "Not published" means we found no figure. Open a profile for the full record.
| Company | What they buy | Price | Deal structure | Status |
|---|---|---|---|---|
| Cloudflare | Publisher content, Web content[1] | Minimum $0.001 USD per crawl[2] | Cloudflare aggregates the charge events, charges the crawler and distributes the earnings to the publisher[3]Pay Per Use: Cloudflare aggregates reported uses, charges the buyer and pays publishers monthly through their connected payment account[4] | Active (closed beta)[1] |
What to check before you sign
- Who sets the price and whether it can differ by page or by use.
- Settlement periods and minimum payout thresholds.
- Which crawlers or uses are charged and which are blocked or allowed free.
- What you need to join, such as hosting or DNS on the provider.
This is general guidance, not legal advice. Your counsel should review any license before you sign.
Sources
- What is Pay Per Crawl? · Cloudflare AI Crawl Control docs, developers.cloudflare.com. Published July 28, 2026. Accessed October 30, 2026. (company statement)
- Set a pay per crawl price · Cloudflare AI Crawl Control docs, developers.cloudflare.com. Published July 28, 2026. Accessed October 30, 2026. (company statement)
- Introducing pay per crawl: Enabling content owners to charge AI crawlers for access | Cloudflare Blog, blog.cloudflare.com. Published July 1, 2025. Accessed October 30, 2026. (company statement)
- Pay Per Use: when AI uses your work, you should get paid, blog.cloudflare.com. Published September 30, 2026. Accessed November 3, 2026. (company statement)
All buyer types and profiles · Data monetization companies compared