Unstructured-IO / unstructured-api

Apache License 2.0
489 stars 102 forks source link

feat: Only allow one Chipper call at a time #296

Closed awalker4 closed 10 months ago

awalker4 commented 10 months ago

Chipper V2 is very memory hungry. While we work to optimize this, we need to restrict the server to one call at a time. While the model is in use, we'll return a 503 "Please try again". Our hosted API should scale up to meet demand, so the next call should route to an available server.

This includes a refactor to how partition_kwargs are passed to either parallel mode, local partition, or local partition with the new Chipper protection.

To verify, try calling Chipper twice:

curl -X POST 'http://localhost:8000/general/v0/general' --form files="@$file" --form strategy=hi_res --form hi_res_model_name=chipper &
curl -X POST 'http://localhost:8000/general/v0/general' --form files="@$file" --form strategy=hi_res --form hi_res_model_name=chipper

The second call will get a 503 response.

Other changes:

sentry-io[bot] commented 10 months ago

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

Did you find this useful? React with a 👍 or 👎