tidyverse / googledrive

Google Drive R API
https://googledrive.tidyverse.org/
Other
323 stars 48 forks source link

Implement retries #380

Open jennybc opened 3 years ago

jennybc commented 3 years ago

I've seen my first quota violation for Drive, via GHA. It looked like this:

── Error (test-drive_examples.R:5:3): drive_examples_remote() lists the remote example files ──
<gargle_error_request_failed/http_error_403/gargle_error/rlang_error/error/condition>
Error: Client error: (403) Forbidden
User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=603366585132
* domain: usageLimits
* reason: userRateLimitExceeded
* message: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=603366585132
* extendedHelp: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=603366585132

https://developers.google.com/drive/api/v3/handle-errors

jennybc commented 1 year ago

This was also ultimately a (very weird form of) quota violation, resulting in a 429 response:

https://github.com/r-lib/gargle/issues/254

It did not look like the one above because it's coming from some opaque non-Drive-API-specific filter that has to do with seeing too much traffic from some range of IP addresses. But if we switch to gargle::request_retry() in googledrive, as we already do in googlesheets4, it should fix or at least make things much better.

jennybc commented 1 year ago

To handle the 403 captured above, gargle::request_retry() needs to gain more functionality. But at least we retry on 429 now.