scalingexcellence / scrapybook

Scrapy Book Code
http://scrapybook.com/
475 stars 209 forks source link

Different spider produces the data, stored in a different database or table #7

Closed xiaowenjie21 closed 8 years ago

xiaowenjie21 commented 8 years ago

If there are two spider, each of item['field'] are not the same, but requires the storage of different tables or databases, how to use different pipelines.py?

lookfwd commented 8 years ago

Hello, You can use custom_settings in your spider to override settings e.g. the destination table/database. An example of overriding settings from the spider can be found here. Would that help?

xiaowenjie21 commented 8 years ago

Thanks