TheAngelBot

What it reads

• On a company's own website: the homepage, up to three of its pricing, blog, changelog or news pages, and its careers page.
• The public job feeds that Greenhouse, Lever and Ashby publish for their customers' job boards. These are read from those providers' own addresses, not from your site, and they are not covered by your robots.txt. The providers publish them for this use.

It reads visible text only. It does not run scripts, load images, log in, fill in forms, or read pages behind a login or a paywall.

How often

Once a week for each company, one request at a time. The homepage is read once in that run. It stops reading a page after 1 MB or 10 seconds, and it follows at most three redirects.

How it behaves

• It reads over https only.
• On your own website, it checks your robots.txt before it reads a page, and again after each redirect. It does not read anything that robots.txt forbids to TheAngelBot.
• It never connects to private or internal network addresses.
• It names itself in every request: TheAngelBot/1.0 (+https://theangel.app/bot)

What it keeps

• For each page of your website: a short fingerprint and an excerpt of at most 500 characters, kept so that a later reading can show what changed.
• For a job board: the number of open roles, and up to 15 job titles of at most 80 characters each.

It does not keep copies of pages. Only the investors who follow the company, and TheAngel staff, can see this.

How to ask it to stop

Add these two lines to your robots.txt. The bot reads robots.txt before every visit to your site, so it stops at its next visit, which is within seven days:

User-agent: TheAngelBot
Disallow: /

Or write to bot@theangel.app with your domain. We stop within five working days, we do not read your site again, and we delete what we hold from reading it.

Who runs it

The Angel App Ltd, registered in England and Wales, company number 09809528, registered office 20 Wenlock Road, London N1 7GU.

Questions

Write to bot@theangel.app.