Admin
Admin token
This server requires the ADMIN_TOKEN it was started with.
When on, every request is checked against the target site's robots.txt first. Sources that would fetch a disallowed page are skipped and listed as "disallowed by robots.txt". Turning this on or off clears cached results, so the next lookups refetch under the new rule.
Worker threads in the company cache crawler. The crawler picks up a new number within a few seconds.
The most requests per second across all sites. Each site and each server IP also has its own lower limit.
When on, the crawler reads each site's robots.txt, skips disallowed pages and follows its crawl-delay. When off, it doesn't read robots.txt at all. This is separate from the switch above, which covers the lookup sources.
When on, no new sites or searches are started. Sites in progress finish.
Company cache
Download contacts
Every email or phone number in the database as a CSV file, with its website, company, category, location and whether it came from a crawl or from Overture. The files are large and take a minute or more to start.
Sources skipped by robots.txt
From the stored results of past lookups.