v7labs / darwin-py

Library and commandline tool for managing datasets on darwin.v7labs.com
MIT License
115 stars 40 forks source link

[DAR-2161][External] Exponential backoff for external storage registration #856

Closed JBWilkie closed 1 month ago

JBWilkie commented 1 month ago

Problem

Recent changes to our external storage registration endpoints mean that they can return HTTP 429 if rate limits are exceeded. This can break registration through darwin-py as it was built assuming this would not be the case

Solution

Introduce exponential backoff using the tenacity library

Changelog

Added exponential backoff for external storage registration

linear[bot] commented 1 month ago

DAR-2161 BUG: Newly added external storage rate limits can break registration through darwin-py in some cases