UCLH-Foundry / PIXL

PIXL Image eXtraction Laboratory
Apache License 2.0
8 stars 0 forks source link

Increase http timeouts because export radiology + parquet upload is performed synchronously #320

Closed jeremyestein closed 5 months ago

jeremyestein commented 6 months ago

The API call to export_patient_data is synchronous. Ie. it has to export the radiology parquet file, and upload all parquet files to FTP before returning via HTTP to the CLI.

Therefore the 10 second timeout on the client side is probably way too short.

I have increased it by a lot here - but the server or something in between may still cause a timeout. If so, we may have to look into sending TCP keepalives. I also increased the orthanc upload by a bit but wasn't sure if that was for a single image or something more.

What’s the biggest set of data this call has been tested with so far? I may be way overestimating how long it takes.

Synchronously waiting for some potentially unreliable tasks (eg FTP upload) to happen is probably not a great design for a distributed system like we have, but that’s a wider problem that should receive separate discussion.