Unable to login using approle

elismaga commented 4 years ago

I am able to use the vault cli and the rest api via curl to use a role id and secret id to get a token, I am also able to login using the token and get a secret. When I use the same role id and secret id with the vault jenkins plugin I get an error that the token is missing. I am running Jenkins 2.159, and Vault plugin version 3.3.0. Vault server version is 1.3.2

The configuration for the vault plugin contains my vault url which I can't disclose but is the same url I used successfully with the CLI. My credential is vault_tools_approle which I will detail below. The vault name space is blank. I have K/V engine version set to "2". I have fail on path unchecked, and I have skip ssl verification unchecked. My vault url is https. My time out is 60.

The vault credential contains the role id and secret id that I used during my tests with the rest api and the cli. The path is set to: v1/auth/app/prod/login The id and and description are set to: vault_tools_approle

This is an example of the curl and CLI commands I used to get a token, login, and then get a secret which works. The role id, secret ID, and url are removed.

curl --request POST --data '{"role_id":"XXXXXX","secret_id":"XXXX"}' https://URL/v1/auth/app/prod/login

vault login TOKEN vault kv get v1/ci/kv/maven/test

In my Jenkins file I have the following code. When I run this it will print "VAULT TEST 1", but does not print the "VAULT TEST 2" and fails. The error and stack trace are below.

   stage('Vault') {
        println("VAULT TEST 1")

        def secrets = [
          [path: 'v1/ci/kv/maven/test', secretValues: [
            [envVar: 'testUser', vaultKey: 'user'],
            [envVar: 'testPassword', vaultKey: 'password']]
          ]
        ]

        withVault([vaultSecrets: secrets]) {
          println("VAULT TEST 2")
          sh 'echo $testUser'
          sh 'echo $testPassword'
        }
    }

com.bettercloud.vault.VaultException: Vault responded with HTTP status code: 400
Response body: {"errors":["missing client token"]}

    at com.bettercloud.vault.api.Auth.loginByAppRole(Auth.java:524)
    at com.datapipe.jenkins.vault.credentials.VaultAppRoleCredential.getToken(VaultAppRoleCredential.java:54)
Caused: com.datapipe.jenkins.vault.exception.VaultPluginException: could not log in into vault
    at com.datapipe.jenkins.vault.credentials.VaultAppRoleCredential.getToken(VaultAppRoleCredential.java:57)
    at com.datapipe.jenkins.vault.credentials.AbstractVaultTokenCredential.authorizeWithVault(AbstractVaultTokenCredential.java:20)
    at com.datapipe.jenkins.vault.VaultAccessor.init(VaultAccessor.java:39)
    at com.datapipe.jenkins.vault.VaultBuildWrapper.provideEnvironmentVariablesFromVault(VaultBuildWrapper.java:148)
    at com.datapipe.jenkins.vault.VaultBuildWrapper.setUp(VaultBuildWrapper.java:95)
    at org.jenkinsci.plugins.workflow.steps.CoreWrapperStep$Execution2.doStart(CoreWrapperStep.java:97)
    at org.jenkinsci.plugins.workflow.steps.GeneralNonBlockingStepExecution.lambda$run$0(GeneralNonBlockingStepExecution.java:77)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

shipilovds commented 4 years ago

Hi there! Any updates? I have the same issue. And I have no idea what to do. Maybe someone knows any workarounds?

elismaga commented 4 years ago

@shipilovds I never heard back from anyone and couldn't figure out the issue. I ended up making some shell scripts to login to vault and get a secret, then created some groovy scripts to help out with running those scripts and creating closure using withEnv. It isn't great but it gets the job done. I will try and get the attention of someone who works on this project to take a look

elismaga commented 4 years ago

@jetersen Any chance you could look into this issue?

shipilovds commented 4 years ago

I have good news: plugin works for me, but I am not sure what exactly helped me. Here is some steps you can try:

delete creds from jenkins
delete vault server settings from global config
delete plugin
restart jenkins
install plugin

make sure your ACL is similar to mine:

path "sys/auth/approle/*" {
capabilities = [ "create", "read", "update", "delete", "sudo" ]
}
path "sys/policies/acl/*" {
capabilities = [ "read", "update", "list" ]
}
path "/auth/token/create" {
capabilities = ["update"]
}
path "kv/data/jenkins" {
capabilities = [ "read", "create", "update", "list" ]
}
path "auth/approle/role/jenkins-example/secret-id" {
capabilities = ["read","create","update"]
}

my approle config for example: (note that secret_id_ttl in my case is 0 to make it permanent)

$ vault read auth/approle/role/jenkins
Key                        Value
---                        -----
bind_secret_id             true
local_secret_ids           false
secret_id_bound_cidrs      <nil>
secret_id_num_uses         0
secret_id_ttl              0s
token_bound_cidrs          []
token_explicit_max_ttl     0s
token_max_ttl              4h
token_no_default_policy    false
token_num_uses             0
token_period               0s
token_policies             [jenkins]
token_ttl                  1h
token_type                 default

update your secret_id
add credentials (make sure Path == 'approle')

And that's it. Check if it works. And please: write here an answer if it helped or not. Maybe I forgot something. If someone can correct me - you are welcome)

jetersen commented 4 years ago

Thanks for sharing, I haven't been able to recreate the reported issue. Most likely because I was using a new instance.

sfc-gh-tvidyasankar commented 4 years ago

I had similar issue.

Only mistake i did from my end was , use the Path in credentials to have actual path of vault instead of add credentials (make sure Path == 'approle') as mentioned by @shipilovds

After i modified Path to "approle" , it worked like charm

sm-powell commented 4 years ago

I had similar issue.

Only mistake i did from my end was , use the Path in credentials to have actual path of vault instead of add credentials (make sure Path == 'approle') as mentioned by @shipilovds

After i modified Path to "approle" , it worked like charm

This was it for me, the 'Path' has to be set correctly.

aajimal commented 4 years ago

I am hitting the same issue even with path set to approle. The work-around mentioned by @shipilovds w/ deleting everything and re-creating seems to work though but this may not be sustainable because all of our vault configuration and credentials are setup through jenkins init.groovy.d scripts on each start.

In my case this is happening when a pipeline is using a vaultUsernamePassword credential with bitbucket-branch-source.

But when I check the credential itself in the UI and and select Test Vault Secrets Retrieval the credential is fetched correctly such as:

2020-08-18 05:49:51.580+0000 [id=266222] INFO c.d.j.v.c.c.VaultHelper$SecretRetrieve#call: Retrieving vault secret path=secret/ci/global key=bitbucket_user engineVersion=2 2020-08-18 05:49:51.580+0000 [id=266222] INFO c.d.j.v.c.common.VaultHelper#retrieveVaultCredentials: Retrieving vault credential ID : global_vault_cicd 2020-08-18 05:49:54.974+0000 [id=266222] INFO c.d.j.v.c.c.VaultHelper$SecretRetrieve#call: Retrieving vault secret path=secret/ci/global key=bitbucket_password engineVersion=2 2020-08-18 05:49:54.975+0000 [id=266222] INFO c.d.j.v.c.common.VaultHelper#retrieveVaultCredentials: Retrieving vault credential ID : global_vault_cicd

Now when I try to use this credential when configuring a pipeline, it will immediately throw an error which looks like it is unable to fetch both keys (username and password):

2020-08-18 05:51:42.583+0000 [id=266470] INFO c.d.j.v.c.c.VaultHelper$SecretRetrieve#call: Retrieving vault secret path=secret/ci/global key=bitbucket_user engineVersion=null 2020-08-18 05:51:42.584+0000 [id=266470] INFO c.d.j.v.c.common.VaultHelper#retrieveVaultCredentials: Retrieving vault credential ID : global_vault_cicd 2020-08-18 05:51:42.584+0000 [id=266470] INFO com.bettercloud.vault.Vault#<init>: Constructing a Vault instance with no provided Engine version, defaulting to version 2. 2020-08-18 05:51:42.655+0000 [id=266470] WARNING o.e.j.s.h.ContextHandler$Context#log: Error while serving https://removed-for-security/job/myjob/descriptorByName/com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource/fillRepositoryItems java.lang.NullPointerException at com.bettercloud.vault.api.Logical.read(Logical.java:75) at com.datapipe.jenkins.vault.VaultAccessor.read(VaultAccessor.java:90) at com.datapipe.jenkins.vault.credentials.common.VaultHelper$SecretRetrieve.call(VaultHelper.java:121) Caused: java.lang.RuntimeException at com.datapipe.jenkins.vault.credentials.common.VaultHelper$SecretRetrieve.call(VaultHelper.java:132) at com.datapipe.jenkins.vault.credentials.common.VaultHelper.getVaultSecret(VaultHelper.java:44) at com.datapipe.jenkins.vault.credentials.common.VaultUsernamePasswordCredentialImpl.getUsername(VaultUsernamePasswordCredentialImpl.java:89) at com.cloudbees.jenkins.plugins.bitbucket.api.credentials.BitbucketUsernamePasswordAuthenticator.<init>(BitbucketUsernamePasswordAuthenticator.java:58) at com.cloudbees.jenkins.plugins.bitbucket.api.credentials.BitbucketUsernamePasswordAuthenticatorSource.convert(BitbucketUsernamePasswordAuthenticatorSource.java:54) at com.cloudbees.jenkins.plugins.bitbucket.api.credentials.BitbucketUsernamePasswordAuthenticatorSource.convert(BitbucketUsernamePasswordAuthenticatorSource.java:36) at jenkins.authentication.tokens.api.AuthenticationTokens.convert(AuthenticationTokens.java:148) at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource$DescriptorImpl.doFillRepositoryItems(BitbucketSCMSource.java:1209) at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:710) at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:396) Caused: java.lang.reflect.InvocationTargetException at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:400) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:408) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:212)

Note: This seems to be happening with other branch sources as well and not just limited to bitbucket-branch-source.

I am using: Jenkins version: 2.235.2 Vault Plugin Version: 3.6.0 Vault Version: 1.3.1

aajimal commented 4 years ago

Update:

I was able to fix this by explicitly setting setEngineVersion() when creating credentials. It seems like, at least in the UI, individual credentials seem to show that they default to 2 but apparently are null which causes that exception after the message:

08-18 05:51:42.584+0000 [id=266470] INFO com.bettercloud.vault.Vault#<init>: Constructing a Vault instance with no provided Engine version, defaulting to version 2. 2020-08-18 05:51:42.655+0000 [id=266470] WARNING

When setting this explicitly on the credential this problem completely went away. Of course this also works if you create the credentials through the UI. So perhaps, when using the API to create credentials via VaultStringCredentialImpl, VaultSSHUserPrivateKeyImpl, or VaultUsernamePasswordCredentialImpl the engine version should be set on construction when not specified since the claim is that the default is always 2.

jetersen commented 4 years ago

Update:

I was able to fix this by explicitly setting setEngineVersion() when creating credentials. It seems like, at least in the UI, individual credentials seem to show that they default to 2 but apparently are null which causes that exception after the message:

08-18 05:51:42.584+0000 [id=266470] INFO com.bettercloud.vault.Vault#<init>: Constructing a Vault instance with no provided Engine version, defaulting to version 2. 2020-08-18 05:51:42.655+0000 [id=266470] WARNING

When setting this explicitly on the credential this problem completely went away. Of course this also works if you create the credentials through the UI. So perhaps, when using the API to create credentials via VaultStringCredentialImpl, VaultSSHUserPrivateKeyImpl, or VaultUsernamePasswordCredentialImpl the engine version should be set on construction when not specified since the claim is that the default is always 2.

@aajimal seems like a separate issue and is being dealt with in #122

I am hitting the same issue even with path set to approle. The work-around mentioned by @shipilovds w/ deleting everything and re-creating seems to work though but this may not be sustainable because all of our vault configuration and credentials are setup through jenkins init.groovy.d scripts on each start.

For your case with init.groovy.d you would add path to be set during inside your groovy for any approle

I have created #123 to hopefully fix the potential null pointer I suspect that people experience.

If your still having issue you have to configure your VaultAppRoleCredential and set the correct auth path for your approles ie. you can have multiple approles on a different paths. See https://www.vaultproject.io/docs/auth/approle

jenkinsci / hashicorp-vault-plugin

Unable to login using approle #97