User Agent: SUMA-EV Web Crawler (in
development)/Nutch-1.15 (We start to build a larger webcrawler. Whilst
in development it might happen that we visit your site more frequently
then normal. But we respect your robots.txt and won't crawl more than
one page every 5 seconds from your domain. If there are any problems
don't hesitate to contact us.; support at suma-ev dot de)
https://nutch.apache.org/
User Agent: SUMA-EV Web Crawler (in development)/Nutch-1.15 (We start to build a larger webcrawler. Whilst in development it might happen that we visit your site more frequently then normal. But we respect your robots.txt and won't crawl more than one page every 5 seconds from your domain. If there are any problems don't hesitate to contact us.; support at suma-ev dot de)