Our crawler
SpeedyHippoBot reads public holiday pages so we can write them up properly. If it is ever a nuisance, this page is how you stop it.
Last updated
How to identify it
Our crawler sends this user-agent string:
SpeedyHippoBot/1.0 (+https://speedyhippo.com/bot)
Anything claiming to be SpeedyHippo that does not match this is not ours, and we would like to know about it.
What it does
It visits public holiday listing and detail pages on travel-company websites and reads the details they publish — the price, the board, the nights, the guests, the departure and what the operator says is included — so we can write the whole deal up on one page and link back to the company selling it.
How politely
- It obeys
robots.txt. If you disallow a path, it does not fetch it. - One request at a time, with a gap of about 1.5 seconds between them to the same site.
- A capped run. At most 250 pages in a single pass, once a day.
- It never touches a booking, enquiry, basket or checkout page, and it never submits a form.
- It does not log in, does not attempt to reach anything behind authentication, and does not execute your site’s JavaScript.
How to block or slow it down
Add a rule for it to your robots.txt:
User-agent: SpeedyHippoBotDisallow: /
Or, to keep it but ask it to slow down, set a Crawl-delay for the same user-agent. We
re-read robots.txt at least daily, so a change takes effect within a day.
Talk to us
If our crawler is causing you a problem, email [email protected] with your domain and roughly when it happened, and we will stop it the same day. You do not need to explain yourself and we will not argue about it.
Anything unclear?
Tell us at [email protected]. We’d rather fix it than argue about it. Anything about a booking — changes, cancellations, payments, protection — goes to Hays Travel, because they hold the booking and we don’t.