search

jschnurr / scrapyscript

Run a Scrapy spider programmatically from a script or a Celery task - no project required.
MIT License
121 stars 26 forks source link

grant additional control to determine what logging settings should be honored #30

Open maxmorlocke opened 3 months ago

maxmorlocke commented 3 months ago

Scrapy will by default attach a new root logger to a CrawlerProcess. There is a controllable boolean present in scrapy these days to configure this, so I've created a passthrough boolean with the same name, etc. to attach to it. Otherwise, celery and scrapy fight over control of logging and make it rather hard to keep settings consistent.