About RankHabitBot
RankHabitBot is the reader RankHabit uses to learn about a website. It only visits a site when someone adds that site to their RankHabit account, so it can suggest topics that fit what the site already covers.
How to recognise it
It identifies itself with this user-agent on every request:
RankHabitBot/1.0 (+https://rankhabit.com/bot)
It doesn't use a fixed list of IP addresses, so go by the name.
What it does
- Reads robots.txt first and follows it, including Disallow rules and Crawl-delay.
- Reads your sitemap, then follows links on your own site. It never leaves your domain.
- Reads up to 200 pages per visit, at most 5 at a time, and gives up on a page after 10 seconds.
- Only reads public pages. It doesn't log in, fill in forms or change anything.
- Keeps page titles, headings, descriptions, a short excerpt, word counts, structured data and internal links. It doesn't keep full page text.
How to block it
Add this to your robots.txt and it will stay away:
User-agent: RankHabitBot Disallow: /
To keep it out of part of your site, list those paths instead of “/”.
How to let it in
If you added your own site to RankHabit and it couldn't be read, one of these is usually the cause:
- A robots.txt rule that blocks all bots. Add a rule for RankHabitBot above it:
User-agent: RankHabitBot Allow: /
- A firewall or bot protection (Cloudflare, your host). Allow requests whose user-agent contains “RankHabitBot”.
- A “coming soon” or password page in front of the site.
Then go back to RankHabit and press “Try again”.
Contact
If RankHabitBot is causing a problem on your site, or you have a question about it, email contact@rankhabit.com. Include your site's address and we'll look into it.