Closed awalker4 closed 10 months ago
This pull request was deployed and Sentry observed the following issues:
/general/v0/general
View Issue/general/v0/general
View Issue/general/v0/general
View Issue/general/v0/general
View Issue/general/v0/general
View IssueDid you find this useful? React with a 👍 or 👎
Chipper V2 is very memory hungry. While we work to optimize this, we need to restrict the server to one call at a time. While the model is in use, we'll return a 503 "Please try again". Our hosted API should scale up to meet demand, so the next call should route to an available server.
This includes a refactor to how partition_kwargs are passed to either parallel mode, local partition, or local partition with the new Chipper protection.
To verify, try calling Chipper twice:
The second call will get a 503 response.
Other changes:
make docker-start-api
for better dev experiencemake docker-start-bash
while we're in here