`IllegalStateException: No MonitoringAction found for xxyz #123` in the console output

yugendarkokkonda commented 1 month ago

Jenkins and plugins versions report

Environment

```text Jenkins: 2.426.1 OS: Linux - 3.10.0-1160.49.1.el7.x86_64 Java: 17.0.9 - Eclipse Adoptium (OpenJDK 64-Bit Server VM) --- analysis-model-api:11.15.0 antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-208.v438351942757 authentication-tokens:1.53.v1c90fd9191a_b_ blueocean:1.27.11 blueocean-bitbucket-pipeline:1.27.11 blueocean-commons:1.27.11 blueocean-config:1.27.11 blueocean-core-js:1.27.11 blueocean-dashboard:1.27.11 blueocean-display-url:2.4.2 blueocean-events:1.27.11 blueocean-git-pipeline:1.27.11 blueocean-github-pipeline:1.27.11 blueocean-i18n:1.27.11 blueocean-jwt:1.27.11 blueocean-personalization:1.27.11 blueocean-pipeline-api-impl:1.27.11 blueocean-pipeline-editor:1.27.11 blueocean-pipeline-scm-api:1.27.11 blueocean-rest:1.27.11 blueocean-rest-impl:1.27.11 blueocean-web:1.27.11 bootstrap5-api:5.3.2-3 bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9 branch-api:2.1152.v6f101e97dd77 caffeine-api:3.1.8-133.v17b_1ff2e0599 checks-api:2.0.2 cloudbees-bitbucket-branch-source:880.vcf4056c5a_71f cloudbees-disk-usage-simple:203.v3f46a_7462b_1a_ cloudbees-folder:6.858.v898218f3609d commons-lang3-api:3.13.0-62.v7d18e55f51e2 commons-text-api:1.11.0-108.v2c41b_8b_2b_269 configuration-as-code:1775.v810dc950b_514 conjur-credentials:1.0.19 credentials:1319.v7eb_51b_3a_c97b_ credentials-binding:657.v2b_19db_7d6e6d data-tables-api:1.13.8-3 display-url-api:2.204.vf6fddd8a_8b_e9 durable-task:555.v6802fe0f0b_82 echarts-api:5.4.3-3 email-ext:2.105 emailext-template:1.5 favorite:2.208.v91d65b_7792a_c font-awesome-api:6.5.1-2 forensics-api:2.3.0 git:5.2.1 git-client:4.7.0 github:1.38.0 github-api:1.318-461.v7a_c09c9fa_d63 github-branch-source:1789.v5b_0c0cea_18c3 gson-api:2.10.1-15.v0d99f670e0a_7 handy-uri-templates-2-api:2.1.8-30.v7e777411b_148 htmlpublisher:1.33 instance-identity:185.v303dc7c645f9 ionicons-api:73.vb_f522f227457 jackson2-api:2.17.0-379.v02de8ec9f64c jacoco:3.3.5 jakarta-activation-api:2.1.3-1 jakarta-mail-api:2.1.3-1 javax-activation-api:1.2.0-6 javax-mail-api:1.6.2-9 jaxb:2.3.9-1 jenkins-design-language:1.27.11 jjwt-api:0.11.5-112.ve82dfb_224b_a_d job-dsl:1.87 jobConfigHistory:1229.v3039470161a_d joda-time-api:2.12.7-29.v5a_b_e3a_82269a_ jquery3-api:3.7.1-1 json-api:20240303-41.v94e11e6de726 json-path-api:2.9.0-33.v2527142f2e1d junit:1265.v65b_14fa_f12f0 kubernetes:4186.v1d804571d5d4 kubernetes-client-api:6.10.0-240.v57880ce8b_0b_2 kubernetes-credentials:0.11 kubernetes-credentials-provider:1.262.v2670ef7ea_0c5 ldap:711.vb_d1a_491714dc mailer:470.vc91f60c5d8e2 matrix-project:822.824.v14451b_c0fd42 metrics:4.2.21-449.v6960d7c54c69 mina-sshd-api-common:2.12.1-101.v85b_e08b_780dd mina-sshd-api-core:2.12.1-101.v85b_e08b_780dd okhttp-api:4.11.0-172.vda_da_1feeb_c6e opentelemetry:3.1111.vc2733c03b_db_1 pipeline-build-step:540.vb_e8849e1a_b_d8 pipeline-graph-analysis:216.vfd8b_ece330ca_ pipeline-groovy-lib:704.vc58b_8890a_384 pipeline-input-step:477.v339683a_8d55e pipeline-milestone-step:119.vdfdc43fc3b_9a_ pipeline-model-api:2.2198.v41dd8ef6dd56 pipeline-model-definition:2.2198.v41dd8ef6dd56 pipeline-model-extensions:2.2198.v41dd8ef6dd56 pipeline-rest-api:2.34 pipeline-stage-step:312.v8cd10304c27a_ pipeline-stage-tags-metadata:2.2198.v41dd8ef6dd56 plain-credentials:179.vc5cb_98f6db_38 plugin-util-api:3.8.0 prism-api:1.29.0-11 prometheus:2.5.1 pubsub-light:1.18 resource-disposer:0.23 role-strategy:689.v731678c3e0eb_ saferestart:0.7 scm-api:683.vb_16722fb_b_80b_ script-security:1336.vf33a_a_9863911 snakeyaml-api:2.2-111.vc6598e30cc65 sse-gateway:1.26 ssh-credentials:337.v395d2403ccd4 sshd:3.303.vefc7119b_ec23 structs:337.v1b_04ea_4df7c8 templating-engine:2.5.3 token-macro:400.v35420b_922dcb_ trilead-api:2.142.v748523a_76693 variant:60.v7290fc0eb_b_cd warnings-ng:10.7.0 workflow-api:1291.v51fd2a_625da_7 workflow-basic-steps:1058.vcb_fc1e3a_21a_9 workflow-cps:3894.3896.vca_2c931e7935 workflow-durable-task-step:1331.vc8c2fed35334 workflow-job:1385.vb_58b_86ea_fff1 workflow-multibranch:773.vc4fe1378f1d5 workflow-scm-step:427.v4ca_6512e7df1 workflow-step-api:657.v03b_e8115821b_ workflow-support:907.v6713a_ed8a_573 ws-cleanup:0.45 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Jenkins is running on Kubernetes pod and currently on Jenkins-core 2.426.1-lts version

Reproduction steps

Install Jenkins 2.426.1. Configure the OpenTelemetry plugin 3 1111 vc2733c03b_db_1 with the following properties:

otel.logs.exporter=otlp
otel.logs.mirror_to_disk=true
otel.instrumentation.jenkins.job.matrix.expand.job.name=true
otel.instrumentation.jenkins.web.enabled=false

Observe that job logs are not visible. Remove the otel.logs.exporter=otlp property. Observe that job logs become visible.

Expected Results

It should the logs as expected without any errors i.e. java.lang.IllegalStateException: No MonitoringAction found for daas_test-new

The OpenTelemetry plugin should work correctly with the configuration property otel.logs.exporter=otlp without causing build log issues that can support Jenkins 2.426.1-lts version.

Actual Results

java.lang.IllegalStateException: No MonitoringAction found for daas_test-new #9
    at io.jenkins.plugins.opentelemetry.job.log.OtelLogStorage.lambda$new$0(OtelLogStorage.java:59)
    at java.base/java.util.Optional.orElseThrow(Optional java: 403)
    at io.jenkins.plugins.opentelemetry.job.log.OtelLogStorage.<init>(OtelLogStorage.java:59)
    at io.jenkins.plugins.opentelemetry.job.log.OtelLogStorageFactory.forBuild(OtelLogStorageFactory.java:73)
    at org-jenkinsci.plugins.workflow.log.LogStorage.of(LogStorage.java:164)
    at org-jenkinsci.plugins.workflow.job.WorkflowRun.getLogText(WorkflowRun.java:1089)
    at hudson.model.Run.writeLogTo(Run.java:1541)
    at jdk.internal.reflect.GeneratedMethodAccessor296149.invoke(Unknown Source)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method. invoke(Method.java: 568)
    at org.apache.commons.jexl.util.introspection.UberspectImp1$VelMethodImpl. invoke(UberspectImpl.java: 258)
    at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
    at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java: 83)
    at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java: 57)
    at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
    at org.apache.commons.jexl.ExpressionImpl.evaluate (ExpressionImpl.java: 80)
    at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:76)
    at org.apache.commons.jelly.parser.EscapingExpression.evaluate(EscapingExpression.java:24)
    at org.apache.commons.jelly.impl.ExpressionScript.run(ExpressionScript.java:66)
    at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
    at org.apache.commons.jelly.tags.core.WhitespaceTag.doTag(WhitespaceTag-java: 48)
    at org.apache.commons.jelly.imp1.TagScript.run(TagScript.java:265)
    at org.apache.commons.jelly.imp1.ScriptBlock.run(ScriptBlock.java:95)
    at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:101)
    at org.apache.commons.jelly.imp1.ScriptBlock.run(ScriptBlock.java:95)
    at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
    at org.apache.commons.jelly.tags.core.OtherwiseTag.doTag(OtherwiseTag.java:41)

error_3 1111 vc2733c03b_db_1

Anything else?

Description: We are experiencing build log issues with various versions of the OpenTelemetry plugin on Jenkins 2.426.1. The issue seems to be related to the configuration property otel.logs.exporter=otlp. When this property is removed, job logs become visible on Jenkins.

Configuration: Jenkins Version: 2.426.1 OpenTelemetry Plugin Versions Tested: 2.15.0: Build log issues 2.16.0: Build log issues 2.17.0: Build log issues 2.18.0: Build log issues 2.19.0: Build log issues 3.1092.va_2a_c52b_dd182: Build log issues 3.1111.vc2733c03b_db_1: Build log issues 3.1135.vdcdb_17548474: Build log issues 3.1138.v80fc844ed246: Build log issues 3.1205.v862c5d236ecc: Build log issues 3.1209.v1d64463d3d6c: Build log issues 3.1215.vc9db_a_0b34c2a: Build log issues 3.1261.v46101e2a_3660: Can see the logs but has plugin dependency issues

Current Workaround: Removing the configuration property otel.logs.exporter=otlp allows job logs to be visible on Jenkins but Opentelemetry is not pushing the logs to LogScale Dashboard.

Additional Information: We are aware that the issue is fixed in OpenTelemetry plugin version 3.1270.v35d71e4855f1, but upgrading Jenkins to a newer version is not feasible for us at this time due to the significant effort required for our teams and builds. We need a bug fix that supports Jenkins 2.426.1-lts version.

Logs and Screenshots: Attached

Impact: This issue is affecting our ability to monitor and debug Jenkins jobs effectively, which is critical for our CI/CD pipeline.

Are you interested in contributing a fix?

No response

cyrille-leclerc commented 1 month ago

Sorry for the inconvenience. Can you please detail what you mean by _"3.1261.v46101e2a3660: Can see the logs but has plugin dependency issues"

yugendarkokkonda commented 1 month ago

@cyrille-leclerc - Thanks for looking into this issue.

After installing the version 3.1261.v46101e2a_3660, I was able to see the older build logs but it has plugin dependency issues and few of the plugins didn't load properly. Please see the attached screenshot of the issue in Manage Jenkins Page.

kuisathaverat commented 2 weeks ago

The plugin is not correctly installed because you do not meet the dependencies; it could be related to many things but not the plugin it self.

jenkinsci / opentelemetry-plugin