Big tech infrastructure

Cloudflare: data programs and licensing terms

Cloudflare’s closed-beta Pay Per Crawl feature charges participating AI crawlers for website access, with Cloudflare acting as merchant of record. This concerns public web content, rather than internal company archives. Reviewed primary source.

Source record date: November 3, 2026

This profile records public-source statements, not a confirmed offer or an independent audit. A listing does not establish that the program is open to your company. Confirm eligibility, rights, privacy duties and the proposed contract directly.

Cloudflare’s cited closed-beta crawl program concerns access to public web content. Beta eligibility, participating crawlers and settlement terms determine potential payment; this does not establish an intake for internal company archives.

Quick facts

Quick facts about Cloudflare
Directory categoryBig tech infrastructure[1]
StatusActive (closed beta)[1]
Websitehttps://www.cloudflare.com/[1]
ProgramPay Per Crawl (feature of AI Crawl Control)[1]https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/[4]

Data described in the sources

Data described in the sources: Cloudflare
Intended usePaid access for AI crawlers[1]

Process

Track record

Related source references; substantive details withheld

Terms to establish before a deal

Date labels identify the retained source record, not a fresh verification of every current term. The gaps below do not establish that a company has no published or private terms. Obtain the applicable written agreement before relying on an offer.

Unverified details and gaps in this source record
  • Data described in the sources: Not independently verified: data types; Not specified in this record: excluded.
  • Who qualifies: Not specified in this record: minimum headcount, minimum history, geography, company stage.
  • Price and payment: Not independently verified: price, pricing basis, payment timing.
  • Deal terms: Not independently verified: deal structure; Not specified in this record: exclusivity, term, field of use, onward transfer, seller keeps ownership.
  • Privacy and data handling: Not specified in this record: de-identification, raw data deletion, published security/compliance statements, personal data handling.
  • Process: Not independently verified: steps; Not specified in this record: timeline, sample before offer.

Watch for

The cited sources do not provide a complete risk assessment. Confirm who receives raw records, what uses are permitted, the payment conditions and deletion duties before granting access.

Sources

  1. What is Pay Per Crawl? · Cloudflare AI Crawl Control docs, developers.cloudflare.com. Recorded publication date July 28, 2026. Recorded access October 30, 2026. (company's own site)
  2. About Cloudflare, cloudflare.com. Recorded access October 30, 2026. (company's own site)
  3. Pay Per Use: when AI uses your work, you should get paid, blog.cloudflare.com. Recorded publication date September 30, 2026. Recorded access November 3, 2026. (company's own site)
  4. Pay Per Crawl · Cloudflare AI Crawl Control docs, developers.cloudflare.com. Recorded access October 30, 2026. (company's own site)
  5. Set a pay per crawl price · Cloudflare AI Crawl Control docs, developers.cloudflare.com. Recorded publication date July 28, 2026. Recorded access October 30, 2026. (company's own site)
  6. Changelog · Cloudflare AI Crawl Control docs, developers.cloudflare.com. Recorded publication date June 16, 2026. Recorded access October 30, 2026. (company's own site)
  7. Manage payouts · Cloudflare AI Crawl Control docs, developers.cloudflare.com. Recorded publication date April 23, 2026. Recorded access October 30, 2026. (company's own site)
  8. Enable in account settings · Cloudflare AI Crawl Control docs, developers.cloudflare.com. Recorded publication date April 23, 2026. Recorded access October 30, 2026. (company's own site)
  9. Introducing pay per crawl: Enabling content owners to charge AI crawlers for access | Cloudflare Blog, blog.cloudflare.com. Recorded publication date July 1, 2025. Recorded access October 30, 2026. (company's own site)
  10. Making AI search smarter | Cloudflare Blog, blog.cloudflare.com. Recorded publication date July 1, 2026. Recorded access October 30, 2026. (company's own site)
  11. Cloudflare’s new policy pushes AI companies to pay for publishers’ content | TechCrunch, techcrunch.com. Recorded publication date July 1, 2026. Recorded access October 30, 2026. (press)

Source record date: November 3, 2026