search

scrapinghub / shub

Scrapinghub Command Line Client
https://shub.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
125 stars 80 forks source link

Support configuring periodic jobs via scrapinghub.yml #220

Open eliasdorneles opened 7 years ago

eliasdorneles commented 7 years ago

It would be nice to be able to configure periodic jobs via the scrapinghub.yml file.

Then, when user does shub deploy, it would sync the periodic jobs configuration from the file with the project periodic jobs.

This would facilitate tracking history of periodic jobs configuration and make the configuration more manageable in many cases.

ejulio commented 5 years ago

In order to add this feature, we'll need to update python-scrapinghub. The commands here make calls to that library (https://github.com/scrapinghub/shub/blob/master/shub/schedule.py#L82)

vshlapakov commented 5 years ago

Let's keep the issue open as a reminder, but it won't happen soon - we don't yet have a stable API for periodic jobs that we could use, which means that it's too early to discuss/plan any changes in the API clients.