Job submission currently blocks middleware

[masonlr]

Actions

Submit a job from the front end. While the job is submitting run the following in a terminal with HTTPie

http GET :5000/job

Expected behaviour

There should be no delay between sending the GET request and receiving a response as it is not related to the submitted job.

Observed behaviour

The GET request returns once the middleware has finished submitting the job. This means that job submission is making the middleware "hang".

[nbarlowATI]

This is presumably because the middleware is waiting for a response from the job manager, which only sends one when it has finished doing the copying/patching.
Maybe we can have the job manager spawn a separate thread to do its thing?
Related to this, it would be better if the job manager posted back to the middleware to say the job is "QUEUED" rather than the middleware setting it by hand. This doesn't work in the current setup because both the job manager and the middleware get stuck waiting to hear back from one another, but if we have a separate process in the middleware it should work.. I can look into this..

[masonlr]

Thanks @nbarlowATI. The threading patch will be great.

Do you if this behaviour is related? When you refresh the frontend it blocks the middleware. See the GIF here:

Edit: Screenshot above uses alan-turing-institute/gateway-frontend@86ad6bf7

Edit: Have observed similar behaviour with alan-turing-institute/gateway-frontend@685349d0

[myyong]

Try npm update in gateway-frontend

[masonlr]

package.json was updated in alan-turing-institute/gateway-frontend@eaef52e0. Newer versions of packages were found to fix the current issue. (@myyong)

[masonlr]

Is there some way that this could be fixed natively? i.e. the middleware's flask development mode could have threading turned on, i.e.:

app.run(debug=True, host='0.0.0.0', port=5000, threaded=True)