search

HumanSignal / label-studio

Label Studio is a multi-type data labeling and annotation tool with standardized output format
https://labelstud.io
Apache License 2.0
18.3k stars 2.3k forks source link

Gateway time-out when `Sync export storage` on large projects #4673

Open WillieMaddox opened 1 year ago

WillieMaddox commented 1 year ago

I am experiencing gateway time-outs on export storage sync requests for some of my larger projects.

This happens regardless of whether I use the web GUI or the API via /api/storages/export/localfiles/{storage_id}/sync. I tried increasing the timeout, but can't seem to go longer than 90 seconds that no matter what I put in the POST request.

Of note, when I sync via the GUI, the syncing process will keep running on the backend even after the 90 second timeout. However, there's nothing in the logs that signal when it's complete. I have to keep calling /api/storages/export?project={project_id} and check if status = completed. Not ideal.

I see two possible solutions to this problem:

  1. Increase the timeout (we should be able to do this; maybe with a flag to the nginx docker?)
  2. When calling from the API, make sync continue on the backend even after it times out (this is a bug).

Thanks,

Willie

jpodivin commented 11 months ago

I have the same problem. Our project has significant number of annotations and we haven't been able to properly sync them with storage.