leendertvanwolfswinkel
commented
9 months ago @ldebek I tested it, and everything worked fine. Obviously, I don't know if it solves the problem, because it is difficult to reproduce. But I think we can assume 95% surely does fix the problem, and it does not break the old workflow.
Please merge
I'm not 100% sure but I think the 'commit' API call is executed directly after the revision files (sqlite, rasters) are uploaded: https://github.com/nens/threedi-api-qgis-client/blob/2629aac9ec39cd89c3b018f0d7ca31fa79e23938/threedi_models_and_simulations/workers.py#L303
This is not a 100% robust approach, if the files are not in 'uploaded' or 'processed' state the 'commit' API call will return an HTTP 400 status with a validation error.
The reason for this is that we use object storage temporary URL's for uploading the files. The files are directly sent to the object storage, without passing the API. However this means that we need to update the file status to 'uploaded' in the API (feedback loop) when the file upload to the object storage is finished. ('processed' is possible because of api-workers that are triggered by the 'uploaded' status update)
Depending on the speed of the feedback loop the commit API call might work or not.
There are 2 ways to mitigate this problem: 1) Retrieve the revision right before the commit API call and check if all files (sqlite & rasters) have 'uploaded' or 'processed' state. This might need to be done in a loop a couple of times with
sleepx seconds. 2) Check the response of the commit API call, if it is an HTTP 400 -> check the body to see if the validation errors are file related.