jenkinsci / checks-api-plugin

Jenkins plugin that defines an API for Jenkins to publish checks to SCM platforms.
https://plugins.jenkins.io/checks-api/
MIT License
43 stars 28 forks source link

Do not report handled exceptions #127

Open notoriousUdo opened 3 years ago

notoriousUdo commented 3 years ago

We're running our pipelines with a Jenkinsfile and a custom groovy-library that is tailored for our infrastructure. In this library we have defined a couple of exceptions to handle such cases to not immediately fail the pipeline

try {
...something that may fail
} catch(Exception e) {
...write to the log and handle the error
}

However, when looking at the GitHub check details these exceptions are still being reported. The picture below shows such a case where the java.nio.file.NoSuchFileException is gracefully handled by our library, but its still being reported as an error:

image

This is confusing for our users.

Version report

Jenkins and plugins versions report:

Jenkins: 2.289.2
OS: Linux - 4.15.0-147-generic
---
ace-editor:1.1
ansicolor:1.0.0
ant:1.11
antisamy-markup-formatter:2.1
apache-httpcomponents-client-4-api:4.5.13-1.0
artifactory:3.12.5
authentication-tokens:1.4
authorize-project:1.4.0
basic-branch-build-strategies:1.3.2
blueocean:1.24.8
blueocean-autofavorite:1.2.4
blueocean-bitbucket-pipeline:1.24.8
blueocean-commons:1.24.8
blueocean-config:1.24.8
blueocean-core-js:1.24.8
blueocean-dashboard:1.24.8
blueocean-display-url:2.4.1
blueocean-events:1.24.8
blueocean-executor-info:1.24.8
blueocean-git-pipeline:1.24.8
blueocean-github-pipeline:1.24.8
blueocean-i18n:1.24.8
blueocean-jira:1.24.8
blueocean-jwt:1.24.8
blueocean-personalization:1.24.8
blueocean-pipeline-api-impl:1.24.8
blueocean-pipeline-editor:1.24.8
blueocean-pipeline-scm-api:1.24.8
blueocean-rest:1.24.8
blueocean-rest-impl:1.24.8
blueocean-web:1.24.8
bootstrap4-api:4.6.0-3
bootstrap5-api:5.1.0-1
bouncycastle-api:2.21
branch-api:2.6.5
build-keeper-plugin:1.3
build-monitor-plugin:1.12+build.201809061734
caffeine-api:2.9.1-23.v51c4e2c879c8
checks-api:1.7.2
cloudbees-bitbucket-branch-source:2.9.10
cloudbees-disk-usage-simple:0.10
cloudbees-folder:6.16
command-launcher:1.6
conditional-buildstep:1.4.1
config-file-provider:3.8.1
configuration-as-code:1.51
copyartifact:1.46.1
credentials:2.5
credentials-binding:1.27
cvs:2.19
dark-theme:0.0.12
display-url-api:2.3.5
docker-commons:1.17
docker-java-api:3.1.5.2
docker-plugin:1.2.2
docker-workflow:1.26
durable-task:1.38
echarts-api:5.1.2-5
email-ext:2.83
extended-read-permission:3.2
external-monitor-job:1.7
favorite:2.3.3
font-awesome-api:5.15.3-4
generic-webhook-trigger:1.74
git:4.8.1
git-client:3.9.0
git-server:1.10
github:1.33.1
github-api:1.123
github-branch-source:2.11.2
github-checks:1.0.13
github-scm-trait-notification-context:1.1
global-slack-notifier:1.5
google-oauth-plugin:1.0.6
gradle:1.37.1
handlebars:3.0.8
handy-uri-templates-2-api:2.1.8-1.0
htmlpublisher:1.25
http_request:1.10
icon-shim:3.0.0
ivy:2.1
jackson2-api:2.12.4
javadoc:1.6
jdk-tool:1.5
jenkins-design-language:1.24.8
jira:3.5
jjwt-api:0.11.2-9.c8b45b8bb173
job-dsl:1.77
jobConfigHistory:2.28.1
jquery:1.12.4-1
jquery-detached:1.2.1
jquery3-api:3.6.0-2
jsch:0.1.55.2
junit:1.51
kubernetes:1.30.1
kubernetes-client-api:5.4.1
kubernetes-credentials:0.9.0
label-linked-jobs:6.0.1
ldap:2.7
lockable-resources:2.11
login-theme:1.1
mailer:1.34
mapdb-api:1.0.9.0
material-theme:0.4.1
matrix-auth:2.6.8
matrix-project:1.19
maven-plugin:3.12
mercurial:2.15
metrics:4.0.2.8
momentjs:1.1.1
multibranch-scan-webhook-trigger:1.0.9
multiple-scms:0.8
next-build-number:1.6
oauth-credentials:0.4
okhttp-api:3.14.9
pam-auth:1.6
parameterized-trigger:2.41
pipeline-build-step:2.15
pipeline-github:2.7
pipeline-graph-analysis:1.11
pipeline-input-step:2.12
pipeline-milestone-step:1.3.2
pipeline-model-api:1.9.1
pipeline-model-definition:1.9.1
pipeline-model-extensions:1.9.1
pipeline-rest-api:2.19
pipeline-stage-step:2.5
pipeline-stage-tags-metadata:1.9.1
pipeline-stage-view:2.19
pipeline-utility-steps:2.8.0
plain-credentials:1.7
plot:2.1.9
plugin-util-api:2.4.0
popper-api:1.16.1-2
popper2-api:2.9.3-1
prometheus:2.0.10
pubsub-light:1.16
purge-build-queue-plugin:1.0
rebuild:1.32
run-condition:1.5
saferestart:0.3
saml:2.0.8
scm-api:2.6.5
script-security:1.78
sidebar-link:1.12.0
simple-theme-plugin:0.7
skip-notifications-trait:1.0.5
slack:2.48
snakeyaml-api:1.29.1
sse-gateway:1.24
ssh:2.6.1
ssh-agent:1.23
ssh-credentials:1.19
ssh-slaves:1.32.0
sshd:3.1.0
structs:1.23
subversion:2.14.4
theme-manager:0.6
token-macro:266.v44a80cf277fd
translation:1.16
trilead-api:1.0.13
variant:1.4
windows-slaves:1.8
workflow-aggregator:2.6
workflow-api:2.46
workflow-basic-steps:2.23
workflow-cps:2.93
workflow-cps-global-lib:2.21
workflow-durable-task-step:2.39
workflow-job:2.41
workflow-multibranch:2.26
workflow-scm-step:2.13
workflow-step-api:2.24
workflow-support:3.8
Controller is unkown (x86+linux)
Agent is RHEL 7.5

Reproduction steps

Results

Expected result:

Check report success and nothing else ( since the pipeline handled the error)

Actual result:

image
KalleOlaviNiemitalo commented 3 years ago

I wonder if checks-api-plugin could perhaps detect whether the exception was caught, by comparing the exceptions of the readFile step to the exceptions propagating out from the try…catch statement. However, the screen shot says "Environment / Read file from workspace", which does not seem to mention the try…catch, so perhaps it is not treated as a step. Should check the source code.

If you make the catch clause use an exception type that does not match what is thrown, what does the output look like then? Does it show the same exception twice: once for the readFile step and again for just "Environment"?

I am surprised that the "name" from stage('name') does not appear in the output at all.

KalleOlaviNiemitalo commented 3 years ago

ExceptionCause might be useful here, for finding the exception that caused the build to fail, or for detecting that an exception did not cause that.

KalleOlaviNiemitalo commented 3 years ago

I tried this and did not see any workflow node for the try…catch. There was no ExceptionCause in build.xml or workflow/*.xml, either.

Looking at the FlowGraphTable in Script Console:

import org.jenkinsci.plugins.workflow.graph.FlowNode
import org.jenkinsci.plugins.workflow.support.visualization.table.FlowGraphTable

Run build = Jenkins.get().getItemByFullName("REDACTED").getBuild("3")
FlowGraphTable table = new FlowGraphTable(build.getExecution())

table.build()

for (FlowGraphTable.Row row : table.getRows()) {
    FlowNode startNode = row.getNode()
    FlowNode endNode = row.endNode // private field :(

    println("depth=${row.getTreeDepth()} ${startNode.getDisplayName()}  ${startNode.getError()?.getDisplayName()}")
    if (endNode != null) {
        println("\t${endNode.getDisplayName()}  ${endNode.getError()?.getDisplayName()}")
    }
}

When FlowExecutionAnalyzer finds an ErrorAction in a table row, it could check all table rows at outer depths. If any of those has an end node that does not have any errors, then consider the error handled and suppress it from the output. Complications:

Alternatives that would not require examining the end nodes:

In any case, the fix would be in FlowExecutionAnalyzer in checks-api-plugin, not in github-checks-plugin.