Mark build as unstable on errors in measurement for single test

[mawHBT]

What feature do you want to see added?

We have a job with successful RTS but in VM-logs of measurements for a specific test you see several exceptions. Logfiles state "BUILD SUCCESSFUL" at the end, are not marked as erroneous and the build is green. This should not happen since you will not recognize the error from "outside". Specific Logfiles should be marked as erroneous and the build should be unstable.

Upstream changes

No response

[bam-hbt]

We tried to reproduce this using the following project: https://github.com/bam-hbt/peass-demo-error But for our build with this pipeline, the build was correctly unstable.

pipeline {
    agent any

    stages {

        stage('checkout') {
            steps {
                dir("$JOB_NAME") {
                    git branch: 'main',
                       url: 'https://github.com/bam-hbt/peass-demo-error'

                }
            }
        }

        stage('gradle build'){
            steps{
                   script {
                        dir("$JOB_NAME") {

                            measure VMs: 2, iterations: 2, repetitions: 5, warmup: 2
                            //executeBeforeClassInMeasurement: true

                        }
                }
            }
        }
    }
}

[bam-hbt]

We could reproduce this now. The problem was, that the test using a mock did not correctly throw its exception. This will be fixed with kopeme 1.3.4. @mawHBT could you test with 1.3.4-SNAPSHOT?

[mawHBT]

After KoPeMe commit 79167e1 and peass-ci-plugin commit b69a5ff a local testbuild is unstable, erroneous measure-logs are marked as such. After some more observation this probably can be closed.

[mawHBT]

This worked for several builds now, so I close this.