Add cache support for IAM token

hermansb commented 4 years ago

Checklist

[x] Tick to sign-off your agreement to the Developer Certificate of Origin (DCO) 1.1
[x] Added tests for code changes or test/build only changes
[x] Updated the change log file (CHANGES.md|CHANGELOG.md) or test/build only changes
[x] Completed the PR template below:

Description

The problem this PR is trying to address is the issue of excessive IAM API key exchanges for access tokens. We share the IAM API key to interact with our cloudant account with all squads in our tribe and we were hitting the rate limit that IAM had set for API key exchanges. Some apps were exchanging tokens more than others, for example our loopback v3 app which exchanged tokens for each cloudant database configured with loopback however there was room for improvement in all apps as each kubernetes pod would request its own token. Approach used in this PR is below

Approach

In order to prevent unnecessary IAM api token exchanges here we provide support for a cache object requiring two methods: get(key) and set(key, value, ttl) where ttl is the time to live. We also provide support for another parameter cacheOffsetInSecs which subtracts from the ttl to store in the cache so that we don't cut it too close to the expiry time. The cache should not return the token if the token is expired and instead allow the library to retrieve a new access token from IAM. The get and set API methods should return promises.

Schema & API Changes

Two new config parameters are now supported in the iamauth plugin. Namely cache and cacheOffsetInSecs (their role is described above)

Security and Privacy

The auth token received from IAM is now stored in a cache on the user side. This creates more responsibility for the library client to ensure the token is not compromised or leaked.

Testing

Added new test supports an IAM token cache in test/plugins/iamauth.js

Monitoring and Logging

Added new log line on line 42 of lib/tokens/IAMTokenManager.js to log when we are using a cached token.

hermansb commented 4 years ago

@dscape @smithsz @ricellis would one of you be able to take a look at this (or poke someone that would be able to)?

ricellis commented 4 years ago

issue of excessive IAM API key exchanges for access tokens

Do you know what version was running when you saw the issue? It looks like the loopback connector is set to 4.2.1 minimum.

Prior to 4.2.2 the client would wait for a token to expire before renewing and then every request would attempt to renew the token simultaneously, which could create a lot of requests at once. That was fixed in 4.2.2 so that there was only 1 request per client instance and the renewal requests are made pre-emptively (at 50% of token lifetime) - so the expectation is one token exchange per client instance every half-hour.

If you still really want to cache tokens between client instances my recommendation would be to consider doing so via setting the iamTokenUrl to point to a proxy where you run a cache. This would have the advantage of making the cache shareable more widely than this approach and usable by other IAM based resources (e.g. not limited to nodejs and Cloudant). Since most other IAM helpers already accept a custom token auth URL this will also make the cache more future-proof. For example, this library will be superseded by an IBM Cloud SDK where these plugin modifications would not be usable, but a cache server could still be used.

ricellis commented 4 years ago

For the record another alternative is to keep a cache independently updated with fresh tokens and use a simpler custom plugin to fetch the token only from the cache and apply it to each request in a Authorization: Bearer <token> header.

cloudant / nodejs-cloudant