search

jenkinsci / jenkinsfile-runner

A command line tool to run Jenkinsfile as a function
MIT License
1.14k stars 296 forks source link

Pipelines may fail if parameter has empty value #165

Closed anbrsap closed 5 years ago

anbrsap commented 5 years ago

Problem

Jenkinsfile Runner has a --arg option to pass parameters to the pipeline. The value of a parameter may be a zero-length string:

/app/bin/jenkinsfile-runner --arg PARAM1='' ...

In such case the respective StringParameterValue in the job's ParametersAction will have its value set to null. This is different from the behavior of a pipeline job with parameters in a classical Jenkins, where the value is never null but a zero-length string if no default value has been defined via the respective StringParameterDefinition). As a consequence pipelines running successfully in a classical Jenkins may break when executed with Jenkinsfile Runner, for instance in hudson.model.StringParameterValue#buildEnvironment()

How to reproduce

Create a directory testcase with a Jenkinsfile having the following contents:

node {
    sh '''\
        #!/bin/bash
        if [ -n "$PARAM1" ]; then
            echo "PARAM1 is set"
        else
            echo "PARAM1 is not set"
        fi
    '''.stripIndent()
}

Change into directory testcase and run the pipeline via Jenkinsfile Runner:

docker run -it \
    -v "$PWD/Jenkinsfile":/workspace/Jenkinsfile \
    -e "JAVA_OPTS=-Dhudson.model.ParametersAction.keepUndefinedParameters=true" \
    ppiper/jenkinsfile-runner \
        --arg PARAM1=

Remark 1: Setting the system property hudson.model.ParametersAction.keepUndefinedParameters=true is required to include all parameters into the environment of pipeline steps like it is done with classical pipeline jobs having expected parameters declared via ParametersDefinitionProperty.

Remark 2: The Docker image ppiper/jenkinsfile-runner may be replaced by your own Jenkinsfile Runner image. You may also run the testcase without Docker if you have a Jenkinsfile Runner available on your machine.

Output:

Started
Running in Durability level: PERFORMANCE_OPTIMIZED
[Pipeline] Start of Pipeline
[Pipeline] node
2019-09-18 08:24:45.970+0000 [id=59]    WARNING o.j.plugins.workflow.cps.DSL#invokeStep: Error storing the arguments for step: sh
java.lang.IllegalArgumentException: Null value not allowed as an environment variable: PARAM1
    at hudson.EnvVars.put(EnvVars.java:378)
    at hudson.model.StringParameterValue.buildEnvironment(StringParameterValue.java:59)
    at hudson.model.ParametersAction.buildEnvironment(ParametersAction.java:145)
    at hudson.model.Run.getEnvironment(Run.java:2383)
    at org.jenkinsci.plugins.workflow.job.WorkflowRun.getEnvironment(WorkflowRun.java:471)
    at org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:72)
    at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:242)
    at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:179)
    at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
    at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
    at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:84)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
    at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:87)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:78)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
    at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
    at com.cloudbees.groovy.cps.Next.step(Next.java:83)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
    at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
    at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
    at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
    at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:186)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:370)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:93)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:282)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:270)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:66)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Running on Jenkins in /tmp/jenkinsfileRunner.tmp/jfr5831395852522203582.run/workspace/job
[Pipeline] {
[Pipeline] sh
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.lang.IllegalArgumentException: Null value not allowed as an environment variable: PARAM1
    at hudson.EnvVars.put(EnvVars.java:378)
    at hudson.model.StringParameterValue.buildEnvironment(StringParameterValue.java:59)
    at hudson.model.ParametersAction.buildEnvironment(ParametersAction.java:145)
    at hudson.model.Run.getEnvironment(Run.java:2383)
    at org.jenkinsci.plugins.workflow.job.WorkflowRun.getEnvironment(WorkflowRun.java:471)
    at org.jenkinsci.plugins.workflow.support.DefaultStepContext.get(DefaultStepContext.java:72)
    at org.jenkinsci.plugins.workflow.steps.StepDescriptor.checkContextAvailability(StepDescriptor.java:264)
    at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:263)
    at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:179)
    at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
    at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1278)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1172)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
    at WorkflowScript.run(WorkflowScript:4)
    at ___cps.transform___(Native Method)
    at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:84)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
    at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:87)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
    at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:78)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
    at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
    at com.cloudbees.groovy.cps.Next.step(Next.java:83)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
    at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
    at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
    at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
    at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
    at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:186)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:370)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:93)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:282)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:270)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:66)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

Expected result:

Started
Running in Durability level: PERFORMANCE_OPTIMIZED
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /build
[Pipeline] {
[Pipeline] sh
PARAM1 is not set
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

Analysis

The Jenkinsfile Runner uses args4j to parse its arguments (see the Bootstrap class).

For the --arg option args4j uses the MapOptionHandler, which converts empty string values to null.

The Jenkinsfile Runner's Runner class passes the parsed key-value pairs from the --arg options - with the value possibly being null - as StringParameterValues in a ParametersAction into the pipeline job.

Proposed Solution

The postConstruct() method of Jenkinsfile Runner's Bootstrap class should replace all null values in map this.workflowParameters by zero-length strings.

oleg-nenashev commented 5 years ago

It was released in https://github.com/jenkinsci/jenkinsfile-runner/releases/tag/1.0-beta-10 . Thanks @anbrsap !

chandrikagudi commented 7 months ago

[Pipeline] Start of Pipeline [Pipeline] node Running on Jenkins in C:\ProgramData\Jenkins.jenkins\workspace\create-issue [Pipeline] { [Pipeline] withEnv [Pipeline] { [Pipeline] stage [Pipeline] { (Deploy Version to Jira) [Pipeline] script [Pipeline] { [Pipeline] jiraGetVersion WARNING: Unknown parameter(s) found for class type 'org.thoughtslive.jenkins.plugins.jira.steps.GetVersionStep': credentialsId,idOrKey Error Code: -1 Error Message: id is empty or null. [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline ERROR: id is empty or null. Finished: FAILURE