Closed elismaga closed 4 years ago
Hi there! Any updates? I have the same issue. And I have no idea what to do. Maybe someone knows any workarounds?
@shipilovds I never heard back from anyone and couldn't figure out the issue. I ended up making some shell scripts to login to vault and get a secret, then created some groovy scripts to help out with running those scripts and creating closure using withEnv. It isn't great but it gets the job done. I will try and get the attention of someone who works on this project to take a look
@jetersen Any chance you could look into this issue?
I have good news: plugin works for me, but I am not sure what exactly helped me. Here is some steps you can try:
path "sys/auth/approle/*" {
capabilities = [ "create", "read", "update", "delete", "sudo" ]
}
path "sys/policies/acl/*" {
capabilities = [ "read", "update", "list" ]
}
path "/auth/token/create" {
capabilities = ["update"]
}
path "kv/data/jenkins" {
capabilities = [ "read", "create", "update", "list" ]
}
path "auth/approle/role/jenkins-example/secret-id" {
capabilities = ["read","create","update"]
}
$ vault read auth/approle/role/jenkins
Key Value
--- -----
bind_secret_id true
local_secret_ids false
secret_id_bound_cidrs <nil>
secret_id_num_uses 0
secret_id_ttl 0s
token_bound_cidrs []
token_explicit_max_ttl 0s
token_max_ttl 4h
token_no_default_policy false
token_num_uses 0
token_period 0s
token_policies [jenkins]
token_ttl 1h
token_type default
And that's it. Check if it works. And please: write here an answer if it helped or not. Maybe I forgot something. If someone can correct me - you are welcome)
Thanks for sharing, I haven't been able to recreate the reported issue. Most likely because I was using a new instance.
I had similar issue.
Only mistake i did from my end was , use the Path in credentials to have actual path of vault instead of add credentials (make sure Path == 'approle') as mentioned by @shipilovds
After i modified Path to "approle" , it worked like charm
I had similar issue.
Only mistake i did from my end was , use the Path in credentials to have actual path of vault instead of add credentials (make sure Path == 'approle') as mentioned by @shipilovds
After i modified Path to "approle" , it worked like charm
This was it for me, the 'Path' has to be set correctly.
I am hitting the same issue even with path set to approle
. The work-around mentioned by @shipilovds w/ deleting everything and re-creating seems to work though but this may not be sustainable because all of our vault configuration and credentials are setup through jenkins init.groovy.d
scripts on each start.
In my case this is happening when a pipeline is using a vaultUsernamePassword
credential with bitbucket-branch-source
.
But when I check the credential itself in the UI and and select Test Vault Secrets Retrieval
the credential is fetched correctly such as:
2020-08-18 05:49:51.580+0000 [id=266222] INFO c.d.j.v.c.c.VaultHelper$SecretRetrieve#call: Retrieving vault secret path=secret/ci/global key=bitbucket_user engineVersion=2 2020-08-18 05:49:51.580+0000 [id=266222] INFO c.d.j.v.c.common.VaultHelper#retrieveVaultCredentials: Retrieving vault credential ID : global_vault_cicd 2020-08-18 05:49:54.974+0000 [id=266222] INFO c.d.j.v.c.c.VaultHelper$SecretRetrieve#call: Retrieving vault secret path=secret/ci/global key=bitbucket_password engineVersion=2 2020-08-18 05:49:54.975+0000 [id=266222] INFO c.d.j.v.c.common.VaultHelper#retrieveVaultCredentials: Retrieving vault credential ID : global_vault_cicd
Now when I try to use this credential when configuring a pipeline, it will immediately throw an error which looks like it is unable to fetch both keys (username and password):
2020-08-18 05:51:42.583+0000 [id=266470] INFO c.d.j.v.c.c.VaultHelper$SecretRetrieve#call: Retrieving vault secret path=secret/ci/global key=bitbucket_user engineVersion=null 2020-08-18 05:51:42.584+0000 [id=266470] INFO c.d.j.v.c.common.VaultHelper#retrieveVaultCredentials: Retrieving vault credential ID : global_vault_cicd 2020-08-18 05:51:42.584+0000 [id=266470] INFO com.bettercloud.vault.Vault#<init>: Constructing a Vault instance with no provided Engine version, defaulting to version 2. 2020-08-18 05:51:42.655+0000 [id=266470] WARNING o.e.j.s.h.ContextHandler$Context#log: Error while serving https://removed-for-security/job/myjob/descriptorByName/com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource/fillRepositoryItems java.lang.NullPointerException at com.bettercloud.vault.api.Logical.read(Logical.java:75) at com.datapipe.jenkins.vault.VaultAccessor.read(VaultAccessor.java:90) at com.datapipe.jenkins.vault.credentials.common.VaultHelper$SecretRetrieve.call(VaultHelper.java:121) Caused: java.lang.RuntimeException at com.datapipe.jenkins.vault.credentials.common.VaultHelper$SecretRetrieve.call(VaultHelper.java:132) at com.datapipe.jenkins.vault.credentials.common.VaultHelper.getVaultSecret(VaultHelper.java:44) at com.datapipe.jenkins.vault.credentials.common.VaultUsernamePasswordCredentialImpl.getUsername(VaultUsernamePasswordCredentialImpl.java:89) at com.cloudbees.jenkins.plugins.bitbucket.api.credentials.BitbucketUsernamePasswordAuthenticator.<init>(BitbucketUsernamePasswordAuthenticator.java:58) at com.cloudbees.jenkins.plugins.bitbucket.api.credentials.BitbucketUsernamePasswordAuthenticatorSource.convert(BitbucketUsernamePasswordAuthenticatorSource.java:54) at com.cloudbees.jenkins.plugins.bitbucket.api.credentials.BitbucketUsernamePasswordAuthenticatorSource.convert(BitbucketUsernamePasswordAuthenticatorSource.java:36) at jenkins.authentication.tokens.api.AuthenticationTokens.convert(AuthenticationTokens.java:148) at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource$DescriptorImpl.doFillRepositoryItems(BitbucketSCMSource.java:1209) at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710) at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:396) Caused: java.lang.reflect.InvocationTargetException at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:400) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:408) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:212)
Note: This seems to be happening with other branch sources as well and not just limited to bitbucket-branch-source.
I am using: Jenkins version: 2.235.2 Vault Plugin Version: 3.6.0 Vault Version: 1.3.1
Update:
I was able to fix this by explicitly setting setEngineVersion()
when creating credentials. It seems like, at least in the UI, individual credentials seem to show that they default to 2
but apparently are null which causes that exception after the message:
08-18 05:51:42.584+0000 [id=266470] INFO com.bettercloud.vault.Vault#<init>: Constructing a Vault instance with no provided Engine version, defaulting to version 2. 2020-08-18 05:51:42.655+0000 [id=266470] WARNING
When setting this explicitly on the credential this problem completely went away. Of course this also works if you create the credentials through the UI. So perhaps, when using the API to create credentials via VaultStringCredentialImpl
, VaultSSHUserPrivateKeyImpl
, or VaultUsernamePasswordCredentialImpl
the engine version should be set on construction when not specified since the claim is that the default is always 2
.
Update:
I was able to fix this by explicitly setting
setEngineVersion()
when creating credentials. It seems like, at least in the UI, individual credentials seem to show that they default to2
but apparently are null which causes that exception after the message:
08-18 05:51:42.584+0000 [id=266470] INFO com.bettercloud.vault.Vault#<init>: Constructing a Vault instance with no provided Engine version, defaulting to version 2. 2020-08-18 05:51:42.655+0000 [id=266470] WARNING
When setting this explicitly on the credential this problem completely went away. Of course this also works if you create the credentials through the UI. So perhaps, when using the API to create credentials via
VaultStringCredentialImpl
,VaultSSHUserPrivateKeyImpl
, orVaultUsernamePasswordCredentialImpl
the engine version should be set on construction when not specified since the claim is that the default is always2
.
@aajimal seems like a separate issue and is being dealt with in #122
I am hitting the same issue even with path set to
approle
. The work-around mentioned by @shipilovds w/ deleting everything and re-creating seems to work though but this may not be sustainable because all of our vault configuration and credentials are setup through jenkinsinit.groovy.d
scripts on each start.
For your case with init.groovy.d
you would add path
to be set during inside your groovy for any approle
I have created #123 to hopefully fix the potential null pointer I suspect that people experience.
If your still having issue you have to configure your VaultAppRoleCredential
and set the correct auth path for your approles ie. you can have multiple approles on a different paths. See https://www.vaultproject.io/docs/auth/approle
I am able to use the vault cli and the rest api via curl to use a role id and secret id to get a token, I am also able to login using the token and get a secret. When I use the same role id and secret id with the vault jenkins plugin I get an error that the token is missing. I am running Jenkins 2.159, and Vault plugin version 3.3.0. Vault server version is 1.3.2
The configuration for the vault plugin contains my vault url which I can't disclose but is the same url I used successfully with the CLI. My credential is vault_tools_approle which I will detail below. The vault name space is blank. I have K/V engine version set to "2". I have fail on path unchecked, and I have skip ssl verification unchecked. My vault url is https. My time out is 60.
The vault credential contains the role id and secret id that I used during my tests with the rest api and the cli. The path is set to: v1/auth/app/prod/login The id and and description are set to: vault_tools_approle
This is an example of the curl and CLI commands I used to get a token, login, and then get a secret which works. The role id, secret ID, and url are removed.
curl --request POST --data '{"role_id":"XXXXXX","secret_id":"XXXX"}' https://URL/v1/auth/app/prod/login
vault login TOKEN vault kv get v1/ci/kv/maven/test
In my Jenkins file I have the following code. When I run this it will print "VAULT TEST 1", but does not print the "VAULT TEST 2" and fails. The error and stack trace are below.