search

opensafely-core / job-server

A server for mediating jobs that can be run in an OpenSAFELY secure environment. q.v. job-runner
https://jobs.opensafely.org
Other
5 stars 11 forks source link

Update job-runner status on GET requests as well as POST #4605

Open bloodearnest opened 1 month ago

bloodearnest commented 1 month ago

There are two main jobs APIs - a POST endpoint for job-runner to provide status on currently running Jobs, and a GET endpoint for currently requested JobRequests. Job-runner polls the GET API regularly, but only polls the POST API when jobs are actually running.

Job runner sends a Flags header with the current status (e.g. db maintenance mode) on every request. However, we only use this to update job-servers status on the POST request: https://github.com/opensafely-core/job-server/blob/main/jobserver/api/jobs.py#L193

This means that when any job is running, and thus job-runner is POSTing updates, we get updated status from the Flags header.

However, if there are no jobs running. we don't use that API, we only GET new job requests instead, the status is not updated.

Short term, we should consider switching the updates to the GET API, so we always have the latest state. Longer term, this API need a bit of a redesign, I think.

evansd commented 1 month ago

That seems like a sensible pragmatic step for now.