Access token caching in Vault

[joeh90]

Is your feature request related to a problem? Please describe. Each request to Vault for a token results in a request to Artifactory to create said token with the given parameters. Heavy plugin usage may therefore result in performance degradation in Artifactory due to a) too many requests to the Access service, b) too many access tokens.

Our personal use case is that we run a compute farm of many thousands of nodes, built on top of Kubernetes and using Armada middleware for batch processing. A user may submit >100,000 jobs over a period of a few seconds, each of which may request an Access token from Artifactory for the same user via the Vault plugin. The possible performance consequences concern us.

Describe the solution you'd like A caching mode (admin-configured). My initial thinking is to override the ttl provided by the user and instead create a token in Artifactory of max_ttl, to be served to the requesting client and stored in Vault for subsequent requests for the same user, up until that token's expiry is <= ttl (or if not provided <= default_ttl), at which point a new token of max_ttl is created and overwritten in the Vault store.

There's an obvious issue here that the user could simply request a token with ttl == max_ttl, therefore evading the caching, so perhaps this could/should be controlled by another parameter.

I'd like to hear your thoughts on this, and am happy to give this a stab if you think it's a welcome addition (otherwise we'll probably maintain a fork).

[alexhung]

@joeh90 I think in your use case, you should consider storing the first/initial token from the plugin in a Vault Cuppyhole (or something similar) so that the token can be reused until expires, and reduces API requests to Artifactory.

We aim to keep the plugin single purpose and not embellishes it with additional features that can be handled by other Vault features/plugins.

[joeh90]

Thanks, I wasn't aware of Cubbyhole. I'll take a look.

[joeh90]

On first glance, it doesn't look like this can be done without adapting the Artifactory plugin itself to write to the cubbyhole. Also, I think the cubbyhole being scoped to a Vault authentication token and with each batch job potentially obtaining a unique token from Vault we wouldn't end up reducing API requests to Artifactory.

[alexhung]

@joeh90 I think you can redirect the plugin output to the Cuppyhole input.

The general idea is that you use the Vault Artifactory plugin to generate a access token, then stores it in another place and uses it until it expires. Then generates another token using the plugin. Whether this storage place is Cuppyhole or some other Vault storage, it's up to you to choose one that fits your circumstances.