microsoft / azure-devops-rust-api

Rust API crate for the Azure DevOps API
MIT License
56 stars 20 forks source link

Add support for Azure DevOps throttling/rate limiting #309

Open johnbatty opened 8 months ago

johnbatty commented 8 months ago

Azure DevOps services provide throttling feedback via response headers:

We have definitions for these headers in the SDK, but currently don't inspect or process them. It would be good to add common function to clients to provide throttling to avoid hitting limits. This should probably be implemented as an azure_core pipeline policy.

It is worth mentioning I recently enhanced the underlying azure_core pipeline to handle retry-after headers when receiving TooManyRequests or ServiceUnavailable responses, and a future release of azure_devops_rust_api will pick this up when it is released. So that will give appropriate backoff/throttling upon hitting the TooManyRequests state. Implementing support for these additional feedback headers would help reduce the chances of hitting the TooManyRequests limit.

Note: I can see that these headers provide feedback, but it is unclear to me what the actual implementation of the processing logic should be. This needs further investigation - including checking whether any of the other SDKs implement this.

Relevant links:

johnbatty commented 8 months ago

Regarding implementation, I found this example which calculates the resource units left in the current window and if this is below a threshold (10% by default), then delays until the window is reset.