Closed Aaron2516 closed 1 month ago
This is more a question for Scrapy than for Scrapyd, since the setup.py file is read by Scrapy, not Scrapyd.
I suggest instead using spider arguments. Your spider's __init__
method can read those arguments, and then reconfigure itself accordingly.
I have a scrapy project with a directory structure similar to the following:
scrapy.cfg
setup.py
When calling the schedule.json, is there a way to tell scrapyd to switch to the settings I need, for example, to start the foo spider use myproject.settings.foo, to start the bar spider use myproject.settings.bar instead of the entry_point field value of setup.py.