search

jenkinsci / github-checks-plugin

Jenkins Plugin for GitHub Checks API
https://plugins.jenkins.io/github-checks/
MIT License
79 stars 38 forks source link

Skip publishing status checks - no effect #189

Open lrozenblyum opened 3 years ago

lrozenblyum commented 3 years ago

Version report

Jenkins and plugins versions report:

Jenkins: 2.289.2 OS: Linux - 4.18.0-301.1.el8.x86_64

git-server:1.10 trilead-api:1.0.13 pam-auth:1.6 jaxb:2.3.0.1 cloudbees-folder:6.15 antisamy-markup-formatter:2.1 jdk-tool:1.5 branch-api:2.6.5 structs:1.23 gradle:1.37.1 workflow-step-api:2.24 token-macro:2.15 handlebars:3.0.8 build-timeout:1.20 credentials:2.5 plain-credentials:1.7 momentjs:1.1.1 ssh-credentials:1.19 credentials-binding:1.27 scm-api:2.6.4 workflow-api:2.46 github-api:1.123 timestamper:1.13 pipeline-stage-view:2.19 script-security:1.77 workflow-support:3.8 durable-task:1.38 workflow-basic-steps:2.23 workflow-durable-task-step:2.39 git:4.7.2 junit:1.51 pipeline-build-step:2.14 command-launcher:1.6 bouncycastle-api:2.21 pipeline-model-api:1.8.5 matrix-project:1.19 resource-disposer:0.16 ws-cleanup:0.39 ant:1.11 github:1.33.1 ace-editor:1.1 jquery-detached:1.2.1 jsch:0.1.55.2 workflow-scm-step:2.13 workflow-cps:2.93 workflow-job:2.41 mailer:1.34 apache-httpcomponents-client-4-api:4.5.13-1.0 display-url-api:2.3.5 pipeline-milestone-step:1.3.2 mapdb-api:1.0.9.0 jackson2-api:2.12.4 git-client:3.8.0 pipeline-input-step:2.12 pipeline-stage-step:2.5 pipeline-rest-api:2.19 pipeline-graph-analysis:1.11 pipeline-model-extensions:1.8.5 workflow-cps-global-lib:2.21 echarts-api:5.1.2-3 workflow-multibranch:2.26 pipeline-stage-tags-metadata:1.8.5 pipeline-model-definition:1.8.5 matrix-auth:2.6.8 plugin-util-api:2.4.0 lockable-resources:2.11 workflow-aggregator:2.6 pipeline-github-lib:1.0 subversion:2.14.4 ssh-slaves:1.32.0 ldap:2.7 jjwt-api:0.11.2-9.c8b45b8bb173 configuration-as-code:1.51 build-name-setter:2.2.0 config-file-provider:3.8.1 popper-api:1.16.1-2 rebuild:1.32 bootstrap4-api:4.6.0-3 jquery:1.12.4-1 git-parameter:0.9.13 mercurial:2.15 handy-uri-templates-2-api:2.1.8-1.0 cloudbees-bitbucket-branch-source:2.9.9 generic-webhook-trigger:1.74 bitbucket-oauth:0.10 jobConfigHistory:2.28.1 snakeyaml-api:1.29.1 ansicolor:1.0.0 jquery3-api:3.6.0-2 github-branch-source:2.11.1 email-ext:2.83 authentication-tokens:1.4 okhttp-api:3.14.9 font-awesome-api:5.15.3-4 cobertura:1.16 forensics-api:1.2.1 maven-plugin:3.12 envinject:2.4.0 javadoc:1.6 pipeline-githubnotify-step:1.0.5 windows-slaves:1.8 uno-choice:2.5.6 checks-api:1.7.2 popper2-api:2.5.4-3 view-job-filters:2.3 block-queued-job:0.2.0 run-condition:1.5 bitbucket-scm-trait-commit-skip:0.4.0 github-checks:1.0.13 code-coverage-api:1.4.0 sshd:3.1.0 text-file-operations:1.3.2 built-on-column:1.1 next-executions:1.0.15 caffeine-api:2.9.1-23.v51c4e2c879c8 bootstrap5-api:5.0.2-1 google-play-android-publisher:4.1 envinject-api:1.7 ssh-agent:1.23 github-oauth:0.33 github-pullrequest:0.3.0 oauth-credentials:0.4 extended-choice-parameter:0.82 docker-commons:1.17 google-oauth-plugin:1.0.6 extensible-choice-parameter:1.7.0 jacoco:3.3.0 data-tables-api:1.10.25-2 pipeline-github:2.7 github-autostatus:3.6.2 htmlpublisher:1.25 docker-custom-build-environment:1.7.3 conditional-buildstep:1.4.1 parameterized-trigger:2.41 external-monitor-job:1.7 jenkins-multijob-plugin:1.36 multiple-scms:0.8 allure-jenkins-plugin:2.29.0 pipeline-multibranch-defaults:2.1 ssh-steps:2.0.0

Reproduction steps

Create a pipeline job with Pipeline script from SCM: Git (GitHub). Add 'Status checks properties' block, with 'Skip publishing status checks' enabled' Run the job

Results

Expected result: No status checks published

Actual result: The status checks are still published.

[GitHub Checks] GitHub check (name: Jenkins, status: IN_PROGRESS) has been published. Running in Durability level: PERFORMANCE_OPTIMIZED .... [GitHub Checks] GitHub check (name: Jenkins, status: COMPLETED) has been published. Finished: FAILURE

KalleOlaviNiemitalo commented 3 years ago

Can you run this in Script Console, changing the foldername/jobname to match your job?

import io.jenkins.plugins.checks.github.status.GitHubStatusChecksProperties
import io.jenkins.plugins.checks.status.BuildStatusChecksPublisher

Job job = Jenkins.get().getItemByFullName("foldername/jobname")

println(new GitHubStatusChecksProperties().isSkipped(job))
println(BuildStatusChecksPublisher.findDeprecatedProperties(job))
println(BuildStatusChecksPublisher.findProperties(job))
println(BuildStatusChecksPublisher.getChecksName(job))

The output should be something like:

true
Optional.empty
io.jenkins.plugins.checks.github.status.GitHubStatusChecksProperties@23fd2ca4
Optional.empty

If BuildStatusChecksPublisher.getChecksName returns Optional.empty, then checks-api-plugin won't publish status checks. If it returns something else, then the first three outputs should help find the reason.

lrozenblyum commented 3 years ago

Hi @KalleOlaviNiemitalo!

The output:

false
Optional.empty
io.jenkins.plugins.checks.github.status.GitHubStatusChecksProperties@445d923a
Optional[Jenkins]

obviously shows the settings are not fine. The corresponding settings block from the Jenkins job (screenshot) image

KalleOlaviNiemitalo commented 3 years ago

The problem may be in SCMFacade.extractFromPipeline:

https://github.com/jenkinsci/github-checks-plugin/blob/aff3a55805a5c1e6c769b59c6a5b5f87dec4b313/src/main/java/io/jenkins/plugins/checks/github/SCMFacade.java#L227-L231

If that returns the wrong SCM (perhaps one used for a pipeline library), then GitHubStatusChecksProperties.getConfigurations might not find the GitHubStatusChecksConfigurations. Then, its caller would use DEFAULT_CONFIGURATION instead, and DefaultGitHubStatusChecksConfigurations.isSkip() would return false.

KalleOlaviNiemitalo commented 3 years ago

I just filed https://github.com/jenkinsci/github-checks-plugin/issues/205 about something related. The GitHub Checks plugin has a bug that can make it enable status checks by default in a non-GitHub project. If you had installed another checks publisher plugin that had a similar bug, then that plugin could likewise enable status checks in a GitHub project even if you have disabled status checks in GitHubStatusChecksProperties. However, because println(new GitHubStatusChecksProperties().isSkipped(job)) output false when you ran it, this cannot be the reason of your problem. Rather, the TODO line in SCMFacade is the likely cause. Do you have pipeline libraries?

lrozenblyum commented 3 years ago

@KalleOlaviNiemitalo about pipeline libraries:

KalleOlaviNiemitalo commented 3 years ago

@lrozenblyum, please try this:

import hudson.plugins.git.GitSCM
import hudson.scm.SCM
import jenkins.triggers.SCMTriggerItem

Job job = Jenkins.get().getItemByFullName("foldername/jobname")
println(job.getClass())

def scms = []
if (job instanceof AbstractProject) {
    println("\tis AbstractProject")
} else if (job instanceof SCMTriggerItem) {
    println("\tis SCMTriggerItem")
    scms = ((SCMTriggerItem) job).getSCMs()
} else {
    println("\tis neither")
}

for (SCM scm : scms) {
    println("${scm} ${scm.getKey()}")
    if (scm instanceof GitSCM) {
        println("\tis GitSCM")
        for (def extension : ((GitSCM) scm).getExtensions()) {
            println("\thas extension ${extension}")
        }
    } else {
        println("\tis not GitSCM")
    }
}

This script displays the SCMs that SCMFacade.extractFromPipeline sees on the Job, and the extensions that GitHubStatusChecksProperties.getConfigurations would find on each GitSCM. The output also includes the URL of each repository, to help you find why each SCM is on the Job. If the script lists two or more SCMs, and the first SCM does not have GitSCMStatusChecksExtension but a later one has, then the TODO line in SCMFacade.extractFromPipeline is causing the problem.

lrozenblyum commented 3 years ago

Hello @KalleOlaviNiemitalo! sorry for long delay.

Here is the output of the script above for the job of interest:

class org.jenkinsci.plugins.workflow.job.WorkflowJob
    is SCMTriggerItem
hudson.plugins.git.GitSCM@34e39b12 git https://github.com/<excluded for privacy>
    is GitSCM
    has extension CleanBeforeCheckout{deleteUntrackedNestedRepositories=false}
timja commented 3 years ago

Try configure it with GitHubSCM instead of Git, might work better

KalleOlaviNiemitalo commented 3 years ago

Is the job using a GitSCMSource (not GitHubSCMSource) with GitSCMChecksExtension attached to that?

It seems a bug to me, if github-checks-plugin allows the extension to be added there but then does not find it during the build.

KalleOlaviNiemitalo commented 3 years ago

Does the xml config file of the job mention "checks" at all? If so, how?

lrozenblyum commented 3 years ago

@KalleOlaviNiemitalo the relevant part of the XML of the job

<definition class="org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition" plugin="workflow-cps@2.93">
<scm class="hudson.plugins.git.GitSCM" plugin="git@4.7.2">
<configVersion>2</configVersion>
<userRemoteConfigs>
<hudson.plugins.git.UserRemoteConfig>
<url>https://github.com/<hid for privacy></url>
<credentialsId><hid for privacy></credentialsId>
</hudson.plugins.git.UserRemoteConfig>
</userRemoteConfigs>
<branches>
<hudson.plugins.git.BranchSpec>
<name>${BRANCH}</name>
</hudson.plugins.git.BranchSpec>
</branches>
<doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
<submoduleCfg class="empty-list"/>
<extensions>
<io.jenkins.plugins.checks.github.status.GitSCMStatusChecksExtension plugin="github-checks@1.0.12">
<skip>true</skip>
<unstableBuildNeutral>false</unstableBuildNeutral>
<name>Jenkins</name>
<suppressLogs>false</suppressLogs>
<skipProgressUpdates>false</skipProgressUpdates>
</io.jenkins.plugins.checks.github.status.GitSCMStatusChecksExtension>
</extensions>
</scm>
<scriptPath><hid for privacy>/ci_scripts/Jenkinsfile_deploy</scriptPath>
<lightweight>false</lightweight>
</definition>
KalleOlaviNiemitalo commented 3 years ago

I don't see the CleanBeforeCheckout extension in this XML.

Might you have run the script on the wrong job?

lrozenblyum commented 3 years ago

@KalleOlaviNiemitalo the job looks to be correct...

It's a pipeline run from a git file itself, The relevant extract from the pipeline script

 stage('Checkout scm') {
            steps {
                checkout([
                    $class: 'GitSCM', 
                    branches: [[name: params.BRANCH ]],
                    doGenerateSubmoduleConfigurations: false, 
                    extensions: [[$class: 'CleanBeforeCheckout']], 
                    submoduleCfg: [], 
                    userRemoteConfigs: [[credentialsId: '...', url: '...']]])
            }
        }
KalleOlaviNiemitalo commented 3 years ago

Oh, I see. SCMFacade.extractFromPipeline in github-checks-plugin tries to find SCMs in two ways:

  1. By calling SCMTriggerItem.getSCMs(), which means WorkflowJob.getSCMs() in your case. This returns the SCMs used during the pipeline, i.e. only the GitSCM from your checkout step.
  2. By calling WorkflowJob.getDefinition() and then CpsScmFlowDefinition.getScm(). This would return the SCM from which the pipeline script itself was retrieved. That SCM has the GitSCMStatusChecksExtension. However, SCMFacade.extractFromPipeline does not get this far because SCMTriggerItem.getSCMs() already returned an SCM.

This could be fixed by making SCMFacade.extractFromPipeline search for SCMs in both ways even if SCMTriggerItem.getSCMs() already found an SCM. Alternatively, you could add the GitSCMStatusChecksExtension to the extensions: list in the checkout step. The latter solution might be cleaner because then the GitSCMStatusChecksExtension would be on the SCM that describes the repository to which the checks are published.

[GitHub Checks] GitHub check (name: Jenkins, status: IN_PROGRESS) has been published. Running in Durability level: PERFORMANCE_OPTIMIZED

Because WorkflowJob.getSCMs() looks at the latest successful or completed build, it can return a non-empty collection here even though the checkout step has not yet been executed during the current build.

lrozenblyum commented 3 years ago

Thanks @KalleOlaviNiemitalo, I'll try the latter suggestion (about adding GitSCMStatusChecksExtension to the extensions: list) and will report about the results.

lrozenblyum commented 3 years ago

Hi @KalleOlaviNiemitalo. I've tried this configuration in the checkout phase extensions: [[$class: 'CleanBeforeCheckout'],[$class: 'GitSCMStatusChecksExtension']], yet the results are pretty the same. The 'Skip' checkbox on UI is on but the results of checks are still published.

Does this checkbox also need programmatical configuration inside the jenkinsfile itself?

KalleOlaviNiemitalo commented 3 years ago

Here you can see GitSCMStatusChecksExtension.skip defaults to false:

https://github.com/jenkinsci/github-checks-plugin/blob/9e2588962c360a17ba4372d8fa541b604843ff5a/src/main/java/io/jenkins/plugins/checks/github/status/GitSCMStatusChecksExtension.java#L18

Perhaps [$class: 'GitSCMStatusChecksExtension', skip: true] would work.

lrozenblyum commented 3 years ago

Hi @KalleOlaviNiemitalo, tried the [$class: 'GitSCMStatusChecksExtension', skip: true]. Unfortunately it had no effect... Still immediately after checkout we have a 'IN_PROGRESS' status published immediately, in the end - the 'COMPLETED' status published.

agaudreault commented 3 years ago

We have the same bug and it is caused by our global library being loaded at the beginning of the build. The SCMFacade will return the Global Library SCM instead of the pipeline ones where the check is configured. The problematic is caused by https://github.com/jenkinsci/github-checks-plugin/issues/189#issuecomment-924187932 and the bug is similar to https://github.com/jenkinsci/checks-api-plugin/issues/122.

Would it be possible to exclude all Global Library SCM ? If I run the following code in the jenkins console, it return my configured Global Library.

def inst = Jenkins.getInstance()
def libs = inst.getDescriptor("org.jenkinsci.plugins.workflow.libs.GlobalLibraries").getLibraries()
for( lib in libs ) {
        println lib.getRetriever().getScm().getRemote()
}

I think the SCMFacade should try to find the best SCM with something like this (pseudocode)

Collection<? extends SCM> scms = ((SCMTriggerItem) job).getSCMs();
if (!scms.isEmpty()) {
    List<SCM> librarySCMs = getLibrarySCMs();
    for(SCM scm : scms) {
        if (!librarySCMs.contains(scm)) {
            return scm;
        }
    }
    return scms.iterator().next(); // Default to first one, GitSCMStatusChecksExtension can still be set to make sure to skip
}

Not sure how to implement something to get all the loaded libraries for a specific run, there are multiple types defined in https://github.com/jenkinsci/workflow-cps-global-lib-plugin/tree/master/src/main/java/org/jenkinsci/plugins/workflow/libs. It has a LibrariesAction, but it does not contain the SCM info.

timja commented 3 years ago

any thoughts @uhafner ?

It makes sense to exclude libraries here, remembering there's also folder libraries as well as global. any sensible PR would be merged I think.

uhafner commented 3 years ago

Hmm, in the git-forensics plugin I did not find a way to identify the correct SCM automatically. The only way was to add an SCM parameter that allows the selection of the SCM that should be used. Would such an approach help?

agaudreault commented 3 years ago

This approach would help for commands like publishChecks as well because we could send the scm hint as parameter, but for the global checks, I think if the config is on the on the job object, it could be retrieved by the ChecksPublisherFactory and also from https://github.com/jenkinsci/checks-api-plugin/blob/master/src/main/java/io/jenkins/plugins/checks/status/BuildStatusChecksPublisher.java#L74.

mwos-sl commented 2 years ago

..or just let override credentialId with an optional argument credentialId. If not provided, the plugin can try to figure this our using scm, otherwise credentialId argument would have a priority.

meiswjn commented 1 year ago

We are also facing this issue when using Git instead of GitHub. If it would help to provide some more info on this, let me know.

russtaylor commented 3 months ago

Has anyone found a workaround for this? We've just started running into this as we're updating our old configs and trying to standardize with shared groovy libraries.