About the Urlbox bot

Urlbox renders web pages into images, PDFs, videos and structured content on behalf of our customers. Requests are customer-initiated: a person or application asks for a specific URL and we load that page once in a real browser to capture it. We are not a search or training crawler. We do not follow links or spider a site.

Renders that identify themselves send this user agent:

Urlbox/1.0 (+https://urlbox.com/bot; web-bot-auth)

User agents are trivially spoofed, so we also sign these requests cryptographically using Web Bot Auth (HTTP Message Signatures, RFC 9421). Each request carries Signature, Signature-Input and Signature-Agent headers. You can verify them against our public key directory:

https://urlbox.com/.well-known/http-message-signatures-directory

A request that claims to be Urlbox but carries no valid signature for that key is not us.

Signed requests honour robots.txt for the product token Urlbox, falling back to the * group. If a URL is disallowed for us, the render fails rather than fetching the page.

Note that Urlbox also serves requests that are not signed, where a customer is capturing a page they own or operate, in the same way a browser would when a person visits it. If you want to stop all Urlbox traffic to your site, contact us below and we will block it at our end.

We fetch the page a customer asked for, plus the subresources that page loads. We do not crawl beyond it, and we honour crawl-delay. If our traffic is causing you problems, tell us and we will throttle or block it.

Email [email protected] about anything on this page, including blocking requests to your site, reporting abuse, or verifying whether traffic was genuinely ours.