cmosig / sentle

Sentinel-1 & Sentinel-2 data cubes at large scale (bigger-than-memory) on any machine with integrated cloud detection, snow masking, harmonization, merging, and temporal composites.
MIT License
23 stars 2 forks source link

implement backoff strategy when hitting rate limit on planetary computer #9

Open cmosig opened 3 months ago

cmosig commented 3 months ago

Something like exponential backoff in time if one hits the rate limit. This ensures that one can just leave the process running until everything is done, even if progress are slow.

Need to check for http code 429. https://github.com/microsoft/PlanetaryComputer/discussions/347#discussioncomment-9572550