jenkinsci / JenkinsPipelineUnit

Framework for unit testing Jenkins pipelines
MIT License
1.55k stars 396 forks source link

Evaluation of pipeline continues after error() is called #684

Closed REBELinBLUE closed 3 days ago

REBELinBLUE commented 2 weeks ago

I was testing a pipeline which calls error and noticed the call stack contained the steps after it, this seemed wrong so I double checked in Jenkins

Using the following pipeline

node('master') {
    stage('foo') {
        echo 'bar'
    }
    stage('bar') {
        error 'blah'
    }
    stage('baz') {
        echo 'fux'
    }
}

I ran it and indeed the "baz" stage does not occur and the output is

[Pipeline] Start of Pipeline
[Pipeline] node
Running on [Jenkins](https://jenkins.example.com/computer/(built-in)/) in /var/jenkins_home/workspace/Experiments/test
[Pipeline] {
[Pipeline] stage
[Pipeline] { (foo)
[Pipeline] echo
bar
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (bar)
[Pipeline] error
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: blah
Finished: FAILURE

But if I run a test on the pipeline script

@Test
void testSample() {
    runScript('test/resources/SamplePipeline.groovy')

    printCallStack()
}

the call stack shows the remaining steps

SamplePipeline.run()
  SamplePipeline.node(master, groovy.lang.Closure)
     SamplePipeline.stage(foo, groovy.lang.Closure)
        SamplePipeline.echo(bar)
     SamplePipeline.stage(bar, groovy.lang.Closure)
        SamplePipeline.error(blah)
     SamplePipeline.stage(baz, groovy.lang.Closure)
        SamplePipeline.echo(fux)

To give an example where this causes problems

I have a pipeline which checks some preconditions and calls error if they are not met, later stages in this pipeline can set the build status to unstable (or even back to success after some scripted retries) so calling assertJobStatusFailure() in the test fails because the later steps are being evaluated and setting the status even though they wouldn't do in a real run

TobiX commented 3 days ago

Duplicate of #265?

nre-ableton commented 3 days ago

Indeed, thanks @TobiX. Let's close this issue in favor of that one.

REBELinBLUE commented 3 days ago

Thanks, sorry I completely missed that one, I searched for a while and never found it 🤦‍♂️