ckan / ckanext-harvest

Remote harvesting extension for CKAN
130 stars 203 forks source link

Idea: Use supervisor for triggering harvest #546

Open pdekraker-epa opened 5 months ago

pdekraker-epa commented 5 months ago

Currently supervisor is the suggested way to run the harvest workers. The recurring triggering of harvests is done via a cron job. Docker is becoming more common for CKAN deployments and cron can require setup to run within the container. An alternative approach that may simplify setup could be to allow harvester to run via supervisor in an event listener mode and receive events like TICK_3600 or TICK_60 to initiate the harvest jobs.