search

ScrapeOps / scrapeops-scrapy-sdk

Scrapy extension that gives you all the scraping monitoring, alerting, scheduling, and data validation you will need straight out of the box.
https://scrapeops.io/
BSD 3-Clause "New" or "Revised" License
37 stars 9 forks source link

Ensure exclusions are set before reading settings #8

Closed rvandam closed 1 year ago

rvandam commented 1 year ago

After some investigation, I discovered that SCRAPEOPS_SETTINGS_EXCLUSION_LIST is not being honored because the settings are getting fetched and cached before the exclusion list is set meaning that only the default exclusions (API_KEY, etc) are honored

josephkearney91 commented 1 year ago

Thanks for spotting that - I'll release a new version update on pypi now