AseomaBot
AseomaBot is the crawler behind Aseoma Site Audit. It visits a site only when one of our users runs an audit of a site they manage, and it checks pages for technical SEO problems.
How to recognise it
It identifies itself with this User-Agent string:
Mozilla/5.0 (compatible; AseomaBot/1.0; +https://aseoma.com/bot)
An audit owner can choose to crawl with a Googlebot user agent to see what Google sees; those requests still come from our servers, not from Google.
What it does
- Reads robots.txt and your XML sitemaps, then follows links on the site being audited.
- Requests HTML pages only (plus a small check of linked URLs); it does not submit forms, log in or execute JavaScript.
- Crawls politely: a few requests per second at most, fewer when your server slows down or answers 429/503, and it honours
Crawl-delayandRetry-After. - Stops at the page limit the audit owner set (usually a few hundred to a few thousand pages).
How to control or block it
AseomaBot respects robots.txt. To keep it out of your whole site:
User-agent: AseomaBot Disallow: /
To slow it down instead:
User-agent: AseomaBot Crawl-delay: 5
Audit owners can switch robots.txt handling off only for audits of their own sites.
Questions or problems
If AseomaBot causes trouble on your server, write to [email protected] with the time and the URLs from your logs and we will look into it the same day.