Drop scrapy dependency, use mechanize, bs4 instead

Tribler / cloudomate

Unpermissioned Open Computer API

GNU Lesser General Public License v3.0

15 stars 16 forks source link

Drop scrapy dependency, use mechanize, bs4 instead #8

Closed Jaapp- closed 7 years ago

Jaapp- commented 7 years ago

As Scrapy didn't offer a non-hacky way to launch multiple spiders sequentially, and because of its weight and its less flexible command line based approach, we've decided to replace Scrapy with the combination of mechanize and beautifulsoup 4.