Add MauiBot to the list of crawlers

allinurl / goaccess

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.

https://goaccess.io

MIT License

18.38k stars 1.11k forks source link

Add MauiBot to the list of crawlers #1088

Open lahwaacz opened 6 years ago

lahwaacz commented 6 years ago

Since #560 is taking too long, can you add MauiBot to the list of crawlers? Full user-agent: MauiBot (crawler.feedback+wc@gmail.com).

DavidSewell commented 6 years ago

Ditto for Scrapy. Recently I have seen an exponential growth in the number of connections using that crawler framework: Scrapy/1.5.0 (+https://scrapy.org)