KeyVault Configuration at Project level and not Global.

[antooro]

Your checklist for this issue

• Jenkins version 2.281

  • Plugin version 2.2

Feature Request

It would be great to add the KeyVault configuration at Project-Level and not in a global scope.

[timja]

How do you mean? what's the use case, what would it look like?

You can do this using withAzureKeyvault currently in pipeline

[antooro]

Using a different keyvault in each project. Different URL i mean.

Thanks for the quick reply

Example:

Right now as i see, you configure the kv URL globally.

It would be nice to have the configuration in project level, so my project1 has project1.keyvault and project2 has project2.keyvault.

[timja]

do you mean in the credential provider? I'm not sure if that would work, there's a similar issue here to allow you to access different vaults using withCredentials https://github.com/jenkinsci/azure-keyvault-plugin/issues/70

but you can already do it in pipeline although not as easily with withAzureKeyvault

[Grokzen]

@timja I might be able to give a scenario that can at least describe what is desired even if there is alternative solutions to this.

So we are in a big company with many hundreds of people. We are working in a small team of a few admin people that provide jenkins as a service for all of the other people in the company. With this we have thousands of jenkins jobs and a very large and complex structure of folders and defined jobs that we administrate to some extent and that is used by the people.

So what we want to do is to store all of our credentials and secrets in several different azure keyvaults where we control, provide and grant what project/folder/jenkins job should have access to what secrets. For example a certain group will get a credential that gives them access to deploy certain resources to azure cloud while other groups should have read-only access to other resources and other groups should have no secrets at all. So the idea here is that due to that we can't really control what individual secret is available to each different job in some way we need this or some other kind of setup/solution where we can pick up multiple azure keyvaults and put them in different scopes so they only expose different secrets to different teams based on our internal setup in the copmany and who should have permissions for what. Having all of the secrets in the global space is nice in most cases but not ideal in our case as we do not want to allow all users of the system to have access to sensetive secrets and non sensetive secrets at the same time and we just have to trust them not to use the secrets they are only supposed to consume and no other.

So one solution is to use the withcredentials and point to a url but then you get into a permission problem because you either need to give permissions through jenkins to access that credentials vault for that job alone but that goes against the whole reason as we need to have double credentials for this.

[timja]

So one solution is to use the withcredentials and point to a url but then you get into a permission problem because you either need to give permissions through jenkins to access that credentials vault for that job alone but that goes against the whole reason as we need to have double credentials for this.

Folder level credentials should allow this to work just fine? Add the credentials at the folder level then access them in the pipeline by overriding the credential ID and key vault url?

[Grokzen]

That is just the point, we want to abstract away the need to have to put in another set of credentials just to be able to fetch another credential from the vault. Is it technically possible? yes kinda. Can we abstract it away with some groovy code? perhaps, but we much rather would just like to be able to bind a credentials vault to work the same way as this plugin already provides for the global level to work the same way for any other level of jobs inside jenkins. You can also dabble in the argument that you will have to invent two different solutions for the same thing. Using the plugin for the global level but use this more hackish way for any other level then global. Sure we can commit to go away from the plugin and just use that wrapped solution, or add this feature to the plugin. We dont really care either or, we just dont want to support multiple solutions for kinda the same thing/feature in our use-case.

[timja]

Thanks for the info, makes sense.

Sounds like a good feature for the plugin

[timja]

@chriskilding just wondering if you have this support in your plugin? (couldn't see it in the README)

[chriskilding]

Would this be the equivalent of supporting secret lookup from multiple AWS accounts?

(In our large company, each product gets its own AWS account - in most cases, one account for staging and one for production. Since Jenkins is used as a cross-account orchestrator, sometimes teams would like Jenkins to reach into another account to use certain secrets. I think this is analogous to the scenario described above?)

[timja]

It’s similar, basically on a per job or folder level you can have another credentials provider. So e.g a team can access their own vault inside of a folder and also the global vault if one is configured.

I’ve got something mostly implemented for folders

[chriskilding]

It sounds like the proposed approach would work for hierarchical setups (product has one account/vault, central infrastructure team has one global account/vault). But in the setups we've seen at our company, we also know of orchestrator jobs that do credentials lookups 'sideways' between sibling accounts.

This meant that originally I looked at allowing an arbitrary number of AWS accounts to be connected to the Secrets Manager plugin. That allows for the hierarchical setup as well as sibling setups. However this brought complications:

1. Name clashes - in AWS secrets are accessed by their simple name, which is not namespaced. (Secret ARNs unfortunately did not work reliably as credential IDs.) We would need some namespacing mechanism in the withCredentials API to get round this. This may be different for Azure if secrets are always referenced in a namespaced way.
2. Ensuring reliability in the face of HTTP errors - connecting the plugin to N AWS accounts would mean N sets of HTTP requests. That's more complicated and error-prone than the standard single-account setup. The CredentialsProvider API design doesn't really handle the combination of caching + error handling for this use case: surfacing an error that secret lookups from some accounts failed (and subsequently retrying the lookups & caching them), while allowing a partial list of successfully retrieved secrets to be available and cached, would be hard within the current design.

So I didn't pursue it - at least not yet - and stuck with the standard single-account approach which I know can be reliably supported.

Do you have any thoughts on handling either of those issues?

[timja]

I was wondering if you could use either the account ID as a prefix or let the user choose an ID for the secret store. Then you could access it as dev/my-secret

we also know of orchestrator jobs that do credentials lookups 'sideways' between sibling accounts.

We do somewhat have that too although we try avoid it.

I think this approach should solve most use cases

[timja]

I've created a pull request which adds support for a folder level credentials provider, would love feedback on if this would work for the scenarios y'all are interested in: https://github.com/jenkinsci/azure-keyvault-plugin/pull/97

Easiest way to test it is to upload this into a Jenkins instance on the Advanced tab: https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/plugins/azure-keyvault/132.v8a77411a24ee/azure-keyvault-132.v8a77411a24ee.hpi

Or you can run a test instance with mvn hpi:run on the branch assuming you have maven and java (8 or 11) installed.

(As you can see in the description there's quite a bit to do before merge but it works now)

[chriskilding]

I prefer the idea of assigning the store an ID / namespace, so that it's a distinct concept, rather than prepending it to credential ID strings.

Ideally this value would correspond to the underlying resource, e.g. it would be the AWS account ID. I don't have an opinion yet on whether we'd want users to be able to redefine that with a custom value.

If a store has its own ID / namespace, this could be usable in the withCredentials or credentials bindings like this:

withCredentials([string(credentialsId: 'newrelic-api-key', namespace: '1234567890', variable: 'NEWRELIC_API_KEY')]) {
        // ...
}

pipeline {
    agent any
    environment {
        NEWRELIC_API_KEY = credentials('newrelic-api-key', namespace: '1234567890')
    }
    // ...
}

[timja]

I prefer the idea of assigning the store an ID / namespace, so that it's a distinct concept, rather than prepending it to credential ID strings.

Likely a nicer approach