search

GSA / data.gov

Main repository for the data.gov service
https://data.gov
Other
547 stars 87 forks source link

Create job/complete route #4760

Open jbrown-xentity opened 1 month ago

jbrown-xentity commented 1 month ago

User Story

In order to run jobs as often as possible, data.gov harvesters want job completion to notify the flask app/controller so that it can start new jobs waiting in the queue.

Acceptance Criteria

Background

In order to not wait for harvester to restart, and to keep jobs moving.

Security Considerations (required)

Want to limit this route to authorized only, if possible.

Sketch

Please note there is a race condition here: when the job makes this post/get request, it is still running. It will wait for a response.

Therefore, we want to give an immediate response, wait 1-2 seconds, and then run the analysis. Otherwise we will find the job still running, and nothing will get kicked off.

jbrown-xentity commented 1 month ago

I would not prioritize this highly, this isn't necessary for MVP.