opensafely-core / job-runner

A client for running jobs in an OpenSAFELY secure environment, requested via job-server (q.v.)
Other
4 stars 5 forks source link

Main loop stops looping in some error conditions #15

Closed sebbacon closed 4 years ago

sebbacon commented 4 years ago

I just saw this, and had to restart the container in order to get the loop running again:

job-runner_1  | 2020-07-09 13:02:55,064 INFO        job#45     Reported error 3 (usage: entrypoint.py [-h] script
job-runner_1  | entrypoint.py: error: unrecognized arguments: /tmp/outputs2/seb-please-delete-me-test-research-master-dummy/input.csv
job-runner_1  |  id job#45) to job server
job-runner_1  | 2020-07-09 13:02:55,065 ERROR       job#45     usage: entrypoint.py [-h] script
job-runner_1  | entrypoint.py: error: unrecognized arguments: /tmp/outputs2/seb-please-delete-me-test-research-master-dummy/input.csv
job-runner_1  | Traceback (most recent call last):
job-runner_1  |   File "/app/runner/__init__.py", line 350, in report_result
job-runner_1  |     job = future.result(
job-runner_1  |   File "/root/.pyenv/versions/3.8.3/lib/python3.8/concurrent/futures/_base.py", line 432, in result
job-runner_1  |     return self.__get_result()
job-runner_1  |   File "/root/.pyenv/versions/3.8.3/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
job-runner_1  |     raise self._exception
job-runner_1  | runner.exceptions.ScriptError: usage: entrypoint.py [-h] script
job-runner_1  | entrypoint.py: error: unrecognized arguments: /tmp/outputs2/seb-please-delete-me-test-research-master-dummy/input.csv
sebbacon commented 4 years ago

Thinking about it, I'm going to guess this is a race condition. Locks are available in the pebble library.

sebbacon commented 4 years ago

The main loop actually quite after an assertion error "The only currently-supported mechanism..."

sebbacon commented 4 years ago

Closed because #40 is v similar