eth-cscs / pyfirecrest

Python wrappers for the FirecREST API
https://pyfirecrest.readthedocs.io
BSD 3-Clause "New" or "Revised" License
10 stars 7 forks source link

Add option to make submit call blocking #125

Open finkandreas opened 2 months ago

finkandreas commented 2 months ago

I find myself often doing something like

jobSubmit = client.submit(...)
while client.poll_active(machine, [jobSubmit['jobid']])[0]['state'] != 'PENDING': 
    time.sleep(3)

Often it does not make sense to continue with anything, until the job actually has started on the remote side, hence the constant polling until the job has started (or failed to start).

It could make sense to make already the call client.submit(..., wait_for_start=True) blocking. Especially in the async interface this could be used.

Then pyfirecrest can decide what a sensible sleeping time is, instead of the user having a too tight sleeping between polling for the job status.

In my usecase I really only want to wait until job start, but maybe some other usecases want to submit and wait until the job finishes. So another idea could be wait_until=f7t.state.RUNNING and wait_until=f7t.state.FINISHED, with the default being wait_until=f7t.state.QUEUED.