search

v7labs / darwin-py

Library and commandline tool for managing datasets on darwin.v7labs.com
MIT License
115 stars 42 forks source link

[DAR-2463][External] Adjusted maximum number of items that can be registered in a single request #857

Closed JBWilkie closed 2 months ago

JBWilkie commented 4 months ago

Problem

Recently, we've encountered significant issues with our item processing queues being overloaded. Part of our strategy to mitigate this is to introduce rate limiting for the registration endpoints and a cap on the maximum number of items that can be registered in a single request. This PR introduces the latter for registration through darwin-py

Solution

Significantly reduce the maximum number of items that can be registered in a single request to 10

Changelog

Adjusted the maximum number of items that can be registered in a single request to 10 to reduce the chance of overloading our processing queues

linear[bot] commented 4 months ago

DAR-2463 Adjust number of items grouped into requests by darwin-py external storage bindings

JBWilkie commented 2 months ago

Closing since we haven't made this change on the BE, and we shouldn't introduce this into the SDK until the BE has been updated