Retry 5xx/429s token service requests in IAM plugin.

cloudant / nodejs-cloudant

Cloudant Node.js client library

Apache License 2.0

255 stars 90 forks source link

Retry 5xx/429s token service requests in IAM plugin. #385

Closed smithsz closed 5 years ago

smithsz commented 5 years ago

Token Service (https://iam.cloud.ibm.com/identity/token) Retry 5xx and 429.
/_iam_session Retry 5xx (endpoint will not return 429 response).

aggarwav commented 5 years ago

Hello @smithsz I am on the IBM DevOps Insights team that reported this issue (case CS0503031)

In 4.1.1 version of the Cloudant library, it appears that once Cloudant library runs into an IAM hiccup, it never recovers. New requests to Cloudant library continue to fail. Thus, we need to restart the impacted pod manually - and this causes serious issues for our service.

Will this fix solve that issue? Thanks.

smithsz commented 5 years ago

Hi @aggarwav,

I hope so. This should make IAM authentication more robust. This is what's new...

We will retry token service requests on 5xx or 429 (up to 3 times with a backoff).
If we still can't get the token then we will attempt to fetch it again on the next request made.
We will only stop attempting to fetch the token on a 401 response from the token service (indicating an incorrect api key).

Thanks, Sam