jenkinsci / hashicorp-vault-plugin

Jenkins plugin to populate environment variables from secrets stored in HashiCorp's Vault.
https://plugins.jenkins.io/hashicorp-vault-plugin/
MIT License
217 stars 143 forks source link

Can't use checkout step inside withVault: "Not running on the Jenkins controller JVM" #313

Closed lenaing closed 1 year ago

lenaing commented 1 year ago

Jenkins and plugins versions report

We used to embed a checkout step in a withVault one, and now it fails with java.lang.IllegalStateException: Not running on the Jenkins controller JVM.

Environment ```text Jenkins: 2.401.3 OS: Linux - 3.10.0-1160.80.1.el7.x86_64 Java: 11.0.19 - Eclipse Adoptium (OpenJDK 64-Bit Server VM) --- Office-365-Connector:4.20.0 ansicolor:1.0.3 ant:497.v94e7d9fffa_b_9 antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-150.v7a_b_9d17134a_5 appcenter:0.11.1 artifactory:3.18.9 audit-trail:333.vb_e1b_b_0f1238c authentication-tokens:1.53.v1c90fd9191a_b_ authorize-project:1.7.1 aws-credentials:191.vcb_f183ce58b_9 aws-java-sdk:1.12.529-406.vdeff15e5817d aws-java-sdk-cloudformation:1.12.529-406.vdeff15e5817d aws-java-sdk-codebuild:1.12.529-406.vdeff15e5817d aws-java-sdk-ec2:1.12.529-406.vdeff15e5817d aws-java-sdk-ecr:1.12.529-406.vdeff15e5817d aws-java-sdk-ecs:1.12.529-406.vdeff15e5817d aws-java-sdk-efs:1.12.529-406.vdeff15e5817d aws-java-sdk-elasticbeanstalk:1.12.529-406.vdeff15e5817d aws-java-sdk-iam:1.12.529-406.vdeff15e5817d aws-java-sdk-kinesis:1.12.529-406.vdeff15e5817d aws-java-sdk-logs:1.12.529-406.vdeff15e5817d aws-java-sdk-minimal:1.12.529-406.vdeff15e5817d aws-java-sdk-secretsmanager:1.12.529-406.vdeff15e5817d aws-java-sdk-sns:1.12.529-406.vdeff15e5817d aws-java-sdk-sqs:1.12.529-406.vdeff15e5817d aws-java-sdk-ssm:1.12.529-406.vdeff15e5817d basic-branch-build-strategies:81.v05e333931c7d blueocean:1.27.6 blueocean-bitbucket-pipeline:1.27.6 blueocean-commons:1.27.6 blueocean-config:1.27.6 blueocean-core-js:1.27.6 blueocean-dashboard:1.27.6 blueocean-display-url:2.4.2 blueocean-events:1.27.6 blueocean-git-pipeline:1.27.6 blueocean-github-pipeline:1.27.6 blueocean-i18n:1.27.6 blueocean-jwt:1.27.6 blueocean-personalization:1.27.6 blueocean-pipeline-api-impl:1.27.6 blueocean-pipeline-editor:1.27.6 blueocean-pipeline-scm-api:1.27.6 blueocean-rest:1.27.6 blueocean-rest-impl:1.27.6 blueocean-web:1.27.6 bootstrap5-api:5.3.0-1 bouncycastle-api:2.29 branch-api:2.1122.v09cb_8ea_8a_724 build-keeper-plugin:19.va_df8a_2c65123 build-monitor-plugin:1.14-744.v35fd6fa_a_26b_2 build-name-setter:2.3.0 build-user-vars-plugin:1.9 build-with-parameters:76.v9382db_f78962 caffeine-api:3.1.8-133.v17b_1ff2e0599 checkmarx:2023.2.6 checks-api:2.0.0 cloudbees-bitbucket-branch-source:825.va_6a_dc46a_f97d cloudbees-disk-usage-simple:182.v62ca_0c992a_f3 cloudbees-folder:6.848.ve3b_fd7839a_81 cloudbees-jenkins-advisor:358.v58972d19b_1f0 command-launcher:90.v669d7ccb_7c31 commons-httpclient3-api:3.1-3 commons-lang3-api:3.13.0-62.v7d18e55f51e2 commons-text-api:1.10.0-68.v0d0b_c439292b_ compress-artifacts:98.vb_20f3c77ddf7 config-file-provider:953.v0432a_802e4d2 configuration-as-code:1670.v564dc8b_982d0 copyartifact:714.v28a_34f8c563f credentials:1271.v54b_1c2c6388a_ credentials-binding:631.v861c06d062b_4 cucumber-reports:5.7.6 data-tables-api:1.13.5-1 display-url-api:2.3.9 docker-commons:439.va_3cb_0a_6a_fb_29 docker-workflow:572.v950f58993843 durable-task:523.va_a_22cf15d5e0 echarts-api:5.4.0-5 email-ext:2.100 extended-read-permission:53.v6499940139e5 favorite:2.4.3 folder-properties:1.2.1 font-awesome-api:6.4.0-2 gatling:1.3.0 git:5.2.0 git-client:4.4.0 git-parameter:0.9.19 github:1.37.3 github-api:1.314-431.v78d72a_3fe4c3 github-branch-source:1732.v3f1889a_c475b_ gitlab-plugin:1.7.15 gradle:2.8.2 groovy:453.vcdb_a_c5c99890 h2-api:11.1.4.199-12.v9f4244395f7a_ handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953 hashicorp-vault-plugin:360.v0a_1c04cf807d htmlpublisher:1.32 http_request:1.16 instance-identity:173.va_37c494ec4e5 ionicons-api:56.v1b_1c8c49374e jackson2-api:2.15.2-350.v0c2f3f8fc595 jakarta-activation-api:2.0.1-3 jakarta-mail-api:2.0.1-3 javadoc:243.vb_b_503b_b_45537 javax-activation-api:1.2.0-6 javax-mail-api:1.6.2-8 jaxb:2.3.8-1 jdk-tool:63.v62d2fd4b_4793 jenkins-design-language:1.27.6 jenkinslint:0.14.0 jersey2-api:2.40-1 jjwt-api:0.11.5-77.v646c772fddb_0 job-dsl:1.84 jquery:1.12.4-1 jquery3-api:3.7.0-1 jsch:0.2.8-65.v052c39de79b_2 junit:1217.v4297208a_a_b_ce kubernetes:3995.v227c16b_675ee kubernetes-client-api:6.4.1-215.v2ed17097a_8e9 kubernetes-credentials:0.10.0 ldap:694.vc02a_69c9787f lockable-resources:1185.v0c528656ce04 mailer:463.vedf8358e006b_ mask-passwords:150.vf80d33113e80 matrix-auth:3.1.10 matrix-project:808.v5a_b_5f56d6966 maven-plugin:3.23 mesos:1.0.0 metrics:4.2.18-442.v02e107157925 mina-sshd-api-common:2.10.0-69.v28e3e36d18eb_ mina-sshd-api-core:2.10.0-69.v28e3e36d18eb_ okhttp-api:4.11.0-157.v6852a_a_fa_ec11 pam-auth:1.10 parameterized-scheduler:1.2 pipeline-aws:1.43 pipeline-build-step:505.v5f0844d8d126-AVENGERS pipeline-graph-analysis:202.va_d268e64deb_3 pipeline-groovy-lib:671.v07c339c842e8 pipeline-input-step:477.v339683a_8d55e pipeline-maven:1322.v9ef317a_3e0a_9 pipeline-milestone-step:111.v449306f708b_7 pipeline-model-api:2.2144.v077a_d1928a_40 pipeline-model-definition:2.2144.v077a_d1928a_40 pipeline-model-extensions:2.2144.v077a_d1928a_40 pipeline-npm:95.v5213efa_9585f pipeline-rest-api:2.33 pipeline-stage-step:305.ve96d0205c1c6 pipeline-stage-tags-metadata:2.2144.v077a_d1928a_40 pipeline-stage-view:2.33 pipeline-utility-steps:2.16.0 piwikanalytics:1.2.0 plain-credentials:143.v1b_df8b_d3b_e48 plugin-util-api:3.3.0 pubsub-light:1.17 rake:1.8.0 rebuild:320.v5a_0933a_e7d61 resource-disposer:0.23 role-strategy:680.v3a_6a_1698b_864 rubyMetrics:1.6.5 run-condition:1.6 saferestart:0.7 schedule-build:502.v9379e178e65b_ scm-api:676.v886669a_199a_a_ script-security:1271.vdede89739a_81 simple-theme-plugin:160.vb_76454b_67900 snakeyaml-api:1.33-95.va_b_a_e3e47b_fa_4 sonar:2.15 sse-gateway:1.26 ssh-agent:333.v878b_53c89511 ssh-credentials:308.ve4497b_ccd8f4 sshd:3.249.v2dc2ea_416e33 structs:325.vcb_307d2a_2782 support-core:1356.vd0f980edfa_46 token-macro:384.vf35b_f26814ec trilead-api:2.84.v72119de229b_7 uno-choice:2.7.2 variant:59.vf075fe829ccb webhook-step:173.vfa_b_93560b_977 workflow-aggregator:596.v8c21c963d92d workflow-api:1259.vb_47f14fffc8a_ workflow-basic-steps:1042.ve7b_140c4a_e0c workflow-cps:3769.v8b_e595e4d40d workflow-durable-task-step:1284.v4fcd365b_75b_e workflow-job:1326.ve643e00e9220 workflow-multibranch:756.v891d88f2cd46 workflow-scm-step:415.v434365564324 workflow-step-api:639.v6eca_cd8c04a_a_ workflow-support:848.v5a_383b_d14921 ws-cleanup:0.45 xray-connector:2.6.1 yaml-axis:0.3.0 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Controller is a Docker Jenkins LTS from jenkins/jenkins:lts. Agent is a Docker Agent built from maven:3-amazoncorretto-11.

Reproduction steps

With this pipeline code:

#!groovy
import hudson.util.Secret
import com.cloudbees.plugins.credentials.CredentialsScope
import com.datapipe.jenkins.vault.credentials.VaultAppRoleCredential

VaultAppRoleCredential getVaultCredential() {
    return new VaultAppRoleCredential(
            CredentialsScope.GLOBAL,
            'test-credential',
            '',
            VAULT_ROLE_ID,
            VAULT_SECRET_ID,
            VAULT_ROLE
    )
}

pipeline {
    agent {
        node {
            label 'corretto11'
        }
    }

    stages {
        stage('Test') {
            steps {
                script {
                    def secrets = [
                        [path: 'testing/test', secretValues: [[envVar: 'test', vaultKey: 'test-test']]]
                    ]

                    def configuration = [vaultCredential: getVaultCredential()]
                    withVault([configuration: configuration, vaultSecrets: secrets]) {
                        sh "echo I want to use my secrets in this block"
                        checkout(
                            [
                                $class                           : 'GitSCM',
                                branches                         : scm.branches,
                                doGenerateSubmoduleConfigurations: false,
                                extensions                       : [[$class: 'LocalBranch']],
                                submoduleCfg                     : [],
                                userRemoteConfigs                : [[credentialsId: 'gitCredentials', url: 'git@gitlab.example.com:lenaing/you-know-for-science.git']]
                            ]
                        )
                        sh "end of my block"
                    }
                }
            }
        }
    }
}

Expected Results

I should see my echoes and the checkout should succeed.

Actual Results

It fails just after the first echo.

[Pipeline] withVault
Retrieving secret: testing/test
[Pipeline] {
[Pipeline] sh
+ echo I want to use my secrets
I want to use my secrets in this block
[Pipeline] checkout
Selected Git installation does not exist. Using Default
The recommended git tool is: NONE
using credential mygitcredential
[Pipeline] }
[Pipeline] // withVault
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from ip/ip:port
        at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
        at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
        at hudson.remoting.Channel.call(Channel.java:1000)
        at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
        at com.sun.proxy.$Proxy95.hasGitRepo(Unknown Source)
        at org.jenkinsci.plugins.gitclient.RemoteGitImpl.hasGitRepo(RemoteGitImpl.java:330)
        at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1207)
        at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1305)
        at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:129)
        at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:97)
        at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:84)
        at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: d45458cb-2d83-4a67-80d6-e6c257d389ee
java.lang.IllegalStateException: Not running on the Jenkins controller JVM
    at jenkins.util.JenkinsJVM.checkJenkinsJVM(JenkinsJVM.java:46)
    at org.jenkinsci.plugins.credentialsbinding.masking.SecretPatterns.getAggregateSecretPattern(SecretPatterns.java:57)
    at com.datapipe.jenkins.vault.log.MaskingConsoleLogFilter.lambda$decorateLogger$0(MaskingConsoleLogFilter.java:43)
    at org.jenkinsci.plugins.credentialsbinding.masking.SecretPatterns$MaskingOutputStream.eol(SecretPatterns.java:93)
    at hudson.console.LineTransformationOutputStream.eol(LineTransformationOutputStream.java:61)
    at hudson.console.LineTransformationOutputStream.write(LineTransformationOutputStream.java:57)
    at hudson.console.LineTransformationOutputStream.write(LineTransformationOutputStream.java:75)
    at java.base/java.io.PrintStream.write(PrintStream.java:559)
    at java.base/sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:233)
    at java.base/sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:312)
    at java.base/sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
    at java.base/java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:181)
    at java.base/java.io.PrintStream.newLine(PrintStream.java:625)
    at java.base/java.io.PrintStream.println(PrintStream.java:883)
    at hudson.model.TaskListener._error(TaskListener.java:88)
    at hudson.model.TaskListener.error(TaskListener.java:123)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.hasGitRepo(CliGitAPIImpl.java:407)
    at hudson.plugins.git.GitAPI.hasGitRepo(GitAPI.java:281)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:924)
    at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:902)
    at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:853)
    at hudson.remoting.UserRequest.perform(UserRequest.java:211)
    at hudson.remoting.UserRequest.perform(UserRequest.java:54)
    at hudson.remoting.Request$2.run(Request.java:377)
    at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:125)
    at java.base/java.lang.Thread.run(Thread.java:829)
Finished: FAILURE

Anything else?

I don't know if this is a Credentials Binding plugin's issue or a Vault's one. If it is the earlier, I'll open an issue on their project.

Kind regards,

icep87 commented 1 year ago

This should be solved by #314

jglick commented 1 year ago

Duplicates #311.