Python3WebSpider / ScrapyUniversal

Scrapy Universal Spider
56 stars 41 forks source link

run multiple universal spider with json configs #12

Open viviov opened 2 years ago

viviov commented 2 years ago

I have multiple json config file, but only one universal spider. If we follow like this settings = get_project_settings() process = CrawlerProcess(settings) process.crawl(jsonname1) process.crawl(jsonname2) process.start() it will throw twisted.internet.error.ReactorAlreadyInstalledError: reactor already installed I think is universal spider create multiple times cause this. how to solve it?