Open huard opened 3 years ago
@cehbrecht @cjauvin
The problem I see with this fix is that it is making a big change to the interface in order to essentially overcome a problem with only a very specific component: writing a NetCDF file in parallel with Dask + multiprocessing.
The ideal would be to understand the problem and find a way to mitigate it at the core, but nobody seems to have a clue (I asked in many places/contexts).
We can keep the current interface as is, and just add new thin interfaces without disrupting existing code.
Description
Due to some weird interactions between dask and PyWPS multiprocessing logic, there is a lock happening when PyWPS runs a process in async mode after having run a process in sync mode. One option is to always run in async mode server side, but have the client "wait" for the server to respond, like in a sync call.
At the moment, the execution mode is set using the
progress
argument, which is not intuitive and mixes two concepts, the progress bar display and the execution mode. I believe we should review our user interface for this, to allow in particular the possibility to send async requests but wait for the response in the client.