GSA / data.gov

Main repository for the data.gov service
https://data.gov
Other
547 stars 87 forks source link

Refactor LoadManager to schedule new jobs in DB #4779

Open btylerburton opened 3 weeks ago

btylerburton commented 3 weeks ago

User Story

In order to manage our scheduled jobs, datagovteam wants the load manager to schedule new CRON jobs based on the harvest source config's desired frequency.

Team has discussed proactively writing the time to kick off the next job for a harvest source at the time that a job is picked up (CKAN does it this way). This would require a new "nextJobScheduled" field added to HarvestJob table.

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

Background

[Any helpful contextual notes or links to artifacts/evidence, if needed]

Security Considerations (required)

[Any security concerns that might be implicated in the change. "None" is OK, just be explicit here!]

Sketch