Open juhanurmi opened 10 years ago
Scrapy: The crawler is developed using Scrapy spider framework [1]. This onion site crawler is called OnionBot[2].
At the moment we have many different test tools for hidden services [3]. It might be cleaner solution to rewrite these tools with Scrapy.
[1] http://scrapy.org/ [2] https://github.com/juhanurmi/ahmia/tree/master/onionbot [3] https://github.com/juhanurmi/ahmia/tree/master/tools
Scrapy: The crawler is developed using Scrapy spider framework [1]. This onion site crawler is called OnionBot[2].
At the moment we have many different test tools for hidden services [3]. It might be cleaner solution to rewrite these tools with Scrapy.
[1] http://scrapy.org/ [2] https://github.com/juhanurmi/ahmia/tree/master/onionbot [3] https://github.com/juhanurmi/ahmia/tree/master/tools