Gitlab plugin reports success on build failure

[josemalonsom]

Issue

The plugin always show success in Gitlab even when the job has failed.

Jenkinsfile:

pipeline {

    agent any

    options {
        gitLabConnection('gitlabhost')
        gitlabCommitStatus(name: 'jenkins')
    }

    triggers {
        gitlab(triggerOnPush: true, triggerOnMergeRequest: true, branchFilterType: 'All')
    }

    stages {

        stage('build') {
            steps {
                sh 'false'
            }
        }
    }

}

Context

• Gitlab plugin version: 1.4.5
• Gitlab version: 8.12.9 and 8.15.8
• Jenkins version: 2.32.3
• Job type: Pipeline and Multibranch pipeline.

Logs & Traces

Jenkins job output:

Started by GitLab push by user.
Checking out git http://gitlabhost/user/foo.git to read Jenkinsfile
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url http://gitlabhost/user/foo.git # timeout=10
Fetching upstream changes from http://gitlabhost/user/foo.git
 > git --version # timeout=10
 > git fetch --tags --progress http://gitlabhost/user/foo.git +refs/heads/*:refs/remotes/origin/* +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/*
skipping resolution of commit remotes/origin/test1, since it originates from another repository
 > git rev-parse refs/remotes/origin/test1^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/test1^{commit} # timeout=10
Checking out Revision 03cb96962630b2b9ea374f5773586ef976edf00e (refs/remotes/origin/test1)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 03cb96962630b2b9ea374f5773586ef976edf00e
 > git rev-list 3ec796cfacbb0ada4150ef50245221ccb0cc80b1 # timeout=10
[Pipeline] properties
WARNING: The properties step will remove all JobPropertys currently configured in this job, either from the UI or from an earlier properties step.
This includes configuration for discarding old builds, parameters, concurrent builds and build triggers.
WARNING: Removing existing job property 'Build triggers'
[Pipeline] node
Running on principal in /var/lib/jenkins/workspace/foo-pipeline
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Declarative: Checkout SCM)
[Pipeline] checkout
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url http://gitlabhost/user/foo.git # timeout=10
Fetching upstream changes from http://gitlabhost/user/foo.git
 > git --version # timeout=10
 > git fetch --tags --progress http://gitlabhost/user/foo.git +refs/heads/*:refs/remotes/origin/* +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/*
skipping resolution of commit remotes/origin/test1, since it originates from another repository
 > git rev-parse refs/remotes/origin/test1^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/test1^{commit} # timeout=10
Checking out Revision 03cb96962630b2b9ea374f5773586ef976edf00e (refs/remotes/origin/test1)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 03cb96962630b2b9ea374f5773586ef976edf00e
[Pipeline] }
[Pipeline] // stage
[Pipeline] gitlabCommitStatus
[Pipeline] {
[Pipeline] stage
[Pipeline] { (build)
[Pipeline] sh
[foo-pipeline] Running shell script
+ false
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // gitlabCommitStatus
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE

Gitlab plugin log:

mar 21, 2017 10:23:47 AM INFORMACIÓN com.dabsquared.gitlabjenkins.webhook.GitLabWebHook getDynamic

WebHook called with url: /project/foo-pipeline

mar 21, 2017 10:23:47 AM INFORMACIÓN com.dabsquared.gitlabjenkins.trigger.handler.AbstractWebHookTriggerHandler handle

foo-pipeline triggered for push.

mar 21, 2017 10:23:55 AM INFORMACIÓN com.dabsquared.gitlabjenkins.util.CommitStatusUpdater retrieveGitlabProjectIds

Retrieving gitlab project ids

mar 21, 2017 10:23:55 AM MÁS DETALLADO com.dabsquared.gitlabjenkins.gitlab.GitLabClientBuilder

Call GitLab:
HTTP method: GET
URL: http://gitlabhost/api/v3/projects/93/repository/commits/03cb96962630b2b9ea374f5773586ef976edf00e
Request headers: [
Accept = [application/json],
PRIVATE-TOKEN = [****FILTERED****]
]

mar 21, 2017 10:23:55 AM MÁS DETALLADO com.dabsquared.gitlabjenkins.gitlab.GitLabClientBuilder

Got response from GitLab:
URL: http://gitlabhost/api/v3/projects/93/repository/commits/03cb96962630b2b9ea374f5773586ef976edf00e
Status: 200 OK
Response headers: [
Cache-Control = [max-age=0, private, must-revalidate],
Connection = [keep-alive],
Content-Length = [527],
Content-Type = [application/json],
Date = [Tue, 21 Mar 2017 09:24:16 GMT],
Etag = [W/"7212abbd17e6ca6c3c4d3d14183b33b6"],
Server = [nginx],
Vary = [Origin],
X-Request-Id = [7665953e-fe88-4113-bd4b-da232df917e2],
X-Runtime = [0.071569]
]
Response body: {
  "id" : "03cb96962630b2b9ea374f5773586ef976edf00e",
  "short_id" : "03cb9696",
  "title" : "test 33",
  "author_name" : "user",
  "author_email" : "user@example.com",
  "created_at" : "2017-03-21T10:24:04.000+01:00",
  "committer_name" : "user",
  "committer_email" : "user@example.com",
  "message" : "test 33\n",
  "parent_ids" : [ "3ec796cfacbb0ada4150ef50245221ccb0cc80b1" ],
  "committed_date" : "2017-03-21T10:24:04.000+01:00",
  "authored_date" : "2017-03-21T10:24:04.000+01:00",
  "stats" : {
    "additions" : 1,
    "deletions" : 0,
    "total" : 1
  },
  "status" : null
}

mar 21, 2017 10:23:55 AM MÁS DETALLADO com.dabsquared.gitlabjenkins.gitlab.GitLabClientBuilder

Call GitLab:
HTTP method: POST
URL: http://gitlabhost/api/v3/projects/93/statuses/03cb96962630b2b9ea374f5773586ef976edf00e
Request headers: [
Accept = [application/json],
Content-Type = [application/x-www-form-urlencoded],
PRIVATE-TOKEN = [****FILTERED****]
]

mar 21, 2017 10:23:55 AM MÁS DETALLADO com.dabsquared.gitlabjenkins.gitlab.GitLabClientBuilder

Got response from GitLab:
URL: http://gitlabhost/api/v3/projects/93/statuses/03cb96962630b2b9ea374f5773586ef976edf00e
Status: 201 Created
Response headers: [
Cache-Control = [max-age=0, private, must-revalidate],
Connection = [keep-alive],
Content-Length = [482],
Content-Type = [application/json],
Date = [Tue, 21 Mar 2017 09:24:16 GMT],
Etag = [W/"0e5c5c82069437a4b69d01a160dd06a9"],
Server = [nginx],
Vary = [Origin],
X-Request-Id = [3afde0cd-a2db-4879-89ab-9a018cc12b5e],
X-Runtime = [0.106030]
]
Response body: {
  "id" : 254,
  "sha" : "03cb96962630b2b9ea374f5773586ef976edf00e",
  "ref" : "test1",
  "status" : "running",
  "name" : "jenkins",
  "target_url" : "http://jenkinshost:8080/job/foo-pipeline/66/",
  "description" : null,
  "created_at" : "2017-03-21T09:24:16.604Z",
  "started_at" : "2017-03-21T09:24:16.616Z",
  "finished_at" : null,
  "allow_failure" : false,
  "author" : {
    "name" : "Jenkins",
    "username" : "jenkins",
    "id" : 19,
    "state" : "active",
    "avatar_url" : "http://gitlabhost/uploads/user/avatar/19/jenkinslogo.png",
    "web_url" : "http://gitlabhost/jenkins"
  }
}

mar 21, 2017 10:23:55 AM INFORMACIÓN com.dabsquared.gitlabjenkins.util.CommitStatusUpdater retrieveGitlabProjectIds

Retrieving gitlab project ids

mar 21, 2017 10:23:55 AM MÁS DETALLADO com.dabsquared.gitlabjenkins.gitlab.GitLabClientBuilder

Call GitLab:
HTTP method: GET
URL: http://gitlabhost/api/v3/projects/93/repository/commits/03cb96962630b2b9ea374f5773586ef976edf00e
Request headers: [
Accept = [application/json],
PRIVATE-TOKEN = [****FILTERED****]
]

mar 21, 2017 10:23:55 AM MÁS DETALLADO com.dabsquared.gitlabjenkins.gitlab.GitLabClientBuilder

Got response from GitLab:
URL: http://gitlabhost/api/v3/projects/93/repository/commits/03cb96962630b2b9ea374f5773586ef976edf00e
Status: 200 OK
Response headers: [
Cache-Control = [max-age=0, private, must-revalidate],
Connection = [keep-alive],
Content-Length = [532],
Content-Type = [application/json],
Date = [Tue, 21 Mar 2017 09:24:17 GMT],
Etag = [W/"49dc1813db9cb31a850e741c4a04f085"],
Server = [nginx],
Vary = [Origin],
X-Request-Id = [3a94aec3-0199-49ac-855f-ee33565ad0af],
X-Runtime = [0.071452]
]
Response body: {
  "id" : "03cb96962630b2b9ea374f5773586ef976edf00e",
  "short_id" : "03cb9696",
  "title" : "test 33",
  "author_name" : "user",
  "author_email" : "user@example.com",
  "created_at" : "2017-03-21T10:24:04.000+01:00",
  "committer_name" : "user",
  "committer_email" : "user@example.com",
  "message" : "test 33\n",
  "parent_ids" : [ "3ec796cfacbb0ada4150ef50245221ccb0cc80b1" ],
  "committed_date" : "2017-03-21T10:24:04.000+01:00",
  "authored_date" : "2017-03-21T10:24:04.000+01:00",
  "stats" : {
    "additions" : 1,
    "deletions" : 0,
    "total" : 1
  },
  "status" : "running"
}

mar 21, 2017 10:23:55 AM MÁS DETALLADO com.dabsquared.gitlabjenkins.gitlab.GitLabClientBuilder

Call GitLab:
HTTP method: POST
URL: http://gitlabhost/api/v3/projects/93/statuses/03cb96962630b2b9ea374f5773586ef976edf00e
Request headers: [
Accept = [application/json],
Content-Type = [application/x-www-form-urlencoded],
PRIVATE-TOKEN = [****FILTERED****]
]

mar 21, 2017 10:23:55 AM MÁS DETALLADO com.dabsquared.gitlabjenkins.gitlab.GitLabClientBuilder

Got response from GitLab:
URL: http://gitlabhost/api/v3/projects/93/statuses/03cb96962630b2b9ea374f5773586ef976edf00e
Status: 201 Created
Response headers: [
Cache-Control = [max-age=0, private, must-revalidate],
Connection = [keep-alive],
Content-Length = [504],
Content-Type = [application/json],
Date = [Tue, 21 Mar 2017 09:24:17 GMT],
Etag = [W/"16d14de298f9ee25be2c598124c1de95"],
Server = [nginx],
Vary = [Origin],
X-Request-Id = [df486a05-cb51-43b4-a9d4-7295675313db],
X-Runtime = [0.069802]
]
Response body: {
  "id" : 254,
  "sha" : "03cb96962630b2b9ea374f5773586ef976edf00e",
  "ref" : "test1",
  "status" : "success",
  "name" : "jenkins",
  "target_url" : "http://jenkinshost:8080/job/foo-pipeline/66/",
  "description" : null,
  "created_at" : "2017-03-21T09:24:16.604Z",
  "started_at" : "2017-03-21T09:24:16.616Z",
  "finished_at" : "2017-03-21T09:24:17.130Z",
  "allow_failure" : false,
  "author" : {
    "name" : "Jenkins",
    "username" : "jenkins",
    "id" : 19,
    "state" : "active",
    "avatar_url" : "http://gitlabhost/uploads/user/avatar/19/jenkinslogo.png",
    "web_url" : "http://gitlabhost/jenkins"
  }
}

Problem description

I have a pipeline with the declarative syntax and I expect that the plugin updates the status in Gitlab accordingly when the build fails but it always sets the status to success.

It seems to be related to https://github.com/jenkinsci/gitlab-plugin/issues/501.

[mstg]

Hello! I had the same problem recently. The solution is very easy. First of all, remove gitlabCommitStatus(name: 'jenkins') from options, then add this anywhere after the top pipeline declaration

post {
    failure {
      updateGitlabCommitStatus name: 'build', state: 'failed'
    }
    success {
      updateGitlabCommitStatus name: 'build', state: 'success'
    }
  }

This will mark your commits accordingly. Hope this helps!

[josemalonsom]

@mstg it works! Thank you very much.

But, before I close the issue: it is supposed to work like that or it's a bug?, the status is set when the job is done so I would expect that the status were set accordingly.

[mstg]

@josemalonsom I think it may be a bug. As it is supposed to update with the correct status (Ref)

If you have several steps for build etc., it just marked as passed immediately for me, but using post and updateGitlabCommitStatus works.

[josemalonsom]

@mstg Ok. I'll use your workaround, it is very helpful, but I'll keep the issue open. Thank you :)

[omehegan]

I'm not using Declarative yet so I'm not sure whether this is a bug or not. @colourmeamused since you added the Declarative support, can you take a look at this issue and let us know what the expected workflow is?

[jurriaanpro]

@mstg This does indeed partly fix the issue, but doing this disables the ability to see statuses like pending, building etc. which is (I believe) enabled by having the gitlabCommitStatus(name: 'jenkins') in options.

Having both of them (in options and post) makes it add multiple build statuses to Gitlab, so that sadly doesn't fix that.

[Zaccc123]

Verified that this is a problem for me as well. Having it in options like the readme will always return success even when job `failed.

I am using @mstg suggestion.

However to have the pending status, i modified it to have this in the first step of my build stage. This would show pending on gitlab. This might work for you as well @broodzak

updateGitlabCommitStatus name: 'jenkins', state: 'pending'

[omehegan]

@colourmeamused ping?

[colourmeamused]

@mstg This does indeed partly fix the issue, but doing this disables the ability to see statuses like pending, building etc. which is (I believe) enabled by having the gitlabCommitStatus(name: 'jenkins') in options.

Yup, I don't have access to the code I used when I updated the README, but I was using a similar workaround.

I had the same line in options to set statuses like pending, building etc. in GitLab. gitlabCommitStatus(name: 'jenkins')

And then I explicitly set the status for each stage in a post block within the stage:

post { failure { updateGitlabCommitStatus name: '<stagename>', state: 'failed' } success { updateGitlabCommitStatus name: '<stagename>', state: 'success' } }

Verbose and ugly but it works. We don't explicitly set status for the job, just use options to give it the label jenkins.

So the GitLab page looks something like this: https://about.gitlab.com/images/8_11/pipelines_mr.png GitLab shows live status of each stage as it builds.

I can dig up my code test it and update the example in the README with screenshots unless there's a better workaround?

[omehegan]

@colourmeamused cool, I get it now. If you can verify and update the README when you have a chance, that would be awesome. Thank you!

[simrobin]

Hello,

Just to be sure that I understood it well, using gitlabCommitStatus(name: 'jenkins') in options will not automatically trigger success or failed state?

Do we need to manually add updateGitlabCommitStatus for success and failure in post hooks in each of our declarative pipeline?

[josemalonsom]

I think it's a bug. Debugging you can see that Jenkins is reporting the build as failed but the plugin is setting the build state as success.

The screenshot is from this line of code https://github.com/jenkinsci/gitlab-plugin/blob/fee5f715ccd5e0e611411cb83dfd0cd838409029/src/main/java/com/dabsquared/gitlabjenkins/util/CommitStatusUpdater.java#L35 and the problem occurs here https://github.com/jenkinsci/gitlab-plugin/blob/fee5f715ccd5e0e611411cb83dfd0cd838409029/src/main/java/com/dabsquared/gitlabjenkins/workflow/GitLabCommitStatusStep.java#L73.

It seems that the method BodyExecutionCallback.onSuccess() is called always that the body has completed successfully, but that only means that no exception was thrown and the build result can be any and it should be checked in order to set the commit status but the code assumes that it's always success

[jgeorgeson]

Looks like my #555 is a duplicate of this. I have a workaround there for getting the pending/running/etc status on a per-stage basis. Put gitlabBuilds() in the options{} to pre-announce all your stages, then use gitlabCommitStatus() in all of those stages. Be careful to add post{} sections to conditionally executed stages to close out the status for them.

[josemalonsom]

@omehegan can you take a look at my comment?

I think that it's a bug and labeling it as doc improvement (as in https://github.com/jenkinsci/gitlab-plugin/issues/555) can be hiding the real problem.

[omehegan]

@josemalonsom I'm not sure your analysis is correct. I think when a Jenkins job first starts, the value of BallColor is inherited from the previous job run. That's why when build 1 fails, and then the job runs a second time, its icon in the UI is still red unless and until it completes successfully. That may be what you are seeing when you trace the execution there. And I think what @jgeorgeson is calling a workaround is just how this was designed to work. I'm still not sure there's any other issue here.

[josemalonsom]

@omehegan I checked it again and if I don’t missing something the result of the screenshot is the actual result of the step execution, it changes in every execution with the expected status of the step, it doesn't show the status of previous executions, the BallColor is a value of the step execution result, you can see it here hudson.model.Result.

Anyway, I can’t reproduce the error anymore because right now it’s working as expected, a failure is updated in Gitlab as a failure. I think that it was some other Jenkins’s plugin that had the problem and in some version it was fixed somehow, I can see debugging that now onSuccess is not called anymore when the step fails. Because Jenkins always installs the newest versions in a clean install I can’t figure out which plugin may have been, maybe one related to the declarative pipeline.

Maybe other person with the same problem can confirm that it’s solved when the plugins are updated, ¿@mstg, @Zaccc123, ...?

[omehegan]

Duplicated by #555.