Closed pgchb closed 1 year ago
Hello @pgchb, we are sorry to hear you are facing a problem.
Please note that we have just added an entry in the FAQ of the plugin on why we don't focus on capabilities that involve disabling the Groovy security sandbox and why we prefer the community to create enhancement request to collaboratively productize secure solutions https://github.com/jenkinsci/opentelemetry-plugin/blob/master/README.md#support-for-disabling-the-groovy-sandbox-and-accessing-the-jenkins-pipeline-logs-apis-while-enabling-the-jenkins-opentelemetry-plugin
Regarding your specific problem, I'm not an expert in troubleshooting groovy script errors as I prefer to reduce as much as possible the quantity of Groovy code in Jenkinsfiles in favour of standard sh scripts. That said, could your problem be related to a NullPointerException directly in your Groovy code rather than an NPE in Java code of the plugin?
I looked at the plugin code ralted to the stack trace and don't understand how an NPE could happen there
Hello @cyrille-leclerc,
thanks for the link to the FAQ. I understand the stated reasons. Thanks for having a look anyway.
I don't think the NPE is related to the groovy code directly, because as soon as I change to "Don't store pipeline logs in Elastic" the pipeline runs successful and prints:
HelloWorld 42
Log of flow node 3: HelloWorld 42
Finished: SUCCESS
I had a closer look at the stacktrace I posted and noticed it seems to be randomly ordered, if viewed in the Jenkins console (e.g. via https://jenkins-hostname.com/job/test-getStepLog/42/console). E.g. the type of exception (java.lang.NullPointerException) seems to randomly appear within the text lines from the stacktrace and this is normally shown before the stacktrace. If I run the pipeline multiple times I always get the exact same text lines, but randomly ordered.
If I view it via 'Pipeline steps' (e.g. https://jenkins-hostname.com/job/test-getStepLog/42/execution/node/3/), it is always ordered in the same (probably correct) way:
java.lang.NullPointerException
at io.jenkins.plugins.opentelemetry.job.log.util.LineIterator$JenkinsHttpSessionLineBytesToLineNumberConverter.getContext(LineIterator.java:67)
at io.jenkins.plugins.opentelemetry.job.log.util.LineIterator$JenkinsHttpSessionLineBytesToLineNumberConverter.putLogBytesToLogLine(LineIterator.java:63)
at io.jenkins.plugins.opentelemetry.job.log.util.LineIteratorInputStream.close(LineIteratorInputStream.java:124)
at org.kohsuke.stapler.framework.io.LargeText$BufferSession.close(LargeText.java:546)
at org.kohsuke.stapler.framework.io.LargeText.writeLogTo(LargeText.java:248)
at hudson.console.AnnotatedLargeText.writeLogTo(AnnotatedLargeText.java:164)
at org.kohsuke.stapler.framework.io.LargeText.writeLogTo(LargeText.java:208)
at hudson.console.AnnotatedLargeText.writeLogTo(AnnotatedLargeText.java:154)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:46)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:136)
at WorkflowScript.getFlowNodeLog(WorkflowScript:20)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:41)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
at WorkflowScript.run(WorkflowScript:32)
at ___cps.transform___(Native Method)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:90)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:116)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:85)
at jdk.internal.reflect.GeneratedMethodAccessor563.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
at com.cloudbees.groovy.cps.Next.step(Next.java:83)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:158)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:152)
at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:136)
at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:275)
at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:152)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:187)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:420)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:95)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:330)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:294)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:30)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:70)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
This strange behavior also happens with this simple pipeline with the groovy sandbox enabled:
echo "hello world"
throw new Exception("this is an exception")
Shall I open another issue for this?
Thanks @pgchb , he is a fix for the NPE. I guess the behavior will be to retrieve the logs of the entire pipeline, not just the logs of the current step.
Is this behavior a problem for your use case?
Thanks for providing a fix for the NPE. Sadly this does not cover my use case as I only need to get the logs from all steps of a specific stage in a big pipeline.
As this is a quite unusual use case, I would totally understand if you close this issue. My plan is anyway to use otel.logs.mirror_to_disk=true
and I think when https://github.com/jenkinsci/opentelemetry-plugin/issues/563 is solved, the above code to retrieve step logs should work again.
Jenkins and plugins versions report
Environment
```text Jenkins: 2.361.2 OS: Linux - 5.4.219-126.411.amzn2.x86_64 --- Parameterized-Remote-Trigger:3.1.6.3 ace-editor:1.1 ansicolor:1.0.2 antisamy-markup-formatter:155.v795fb_8702324 anything-goes-formatter:1.0 apache-httpcomponents-client-4-api:4.5.13-138.v4e7d9a_7b_a_e61 authentication-tokens:1.4 badge:1.9.1 basic-branch-build-strategies:71.vc1421f89888e blueocean:1.25.8 blueocean-autofavorite:1.2.5 blueocean-bitbucket-pipeline:1.25.8 blueocean-commons:1.25.8 blueocean-config:1.25.8 blueocean-core-js:1.25.8 blueocean-dashboard:1.25.8 blueocean-display-url:2.4.1 blueocean-events:1.25.8 blueocean-git-pipeline:1.25.8 blueocean-github-pipeline:1.25.8 blueocean-i18n:1.25.8 blueocean-jwt:1.25.8 blueocean-personalization:1.25.8 blueocean-pipeline-api-impl:1.25.8 blueocean-pipeline-editor:1.25.8 blueocean-pipeline-scm-api:1.25.8 blueocean-rest:1.25.8 blueocean-rest-impl:1.25.8 blueocean-web:1.25.8 bootstrap5-api:5.2.1-3 bouncycastle-api:2.26 branch-api:2.1051.v9985666b_f6cc caffeine-api:2.9.3-65.v6a_47d0f4d1fe checks-api:1.8.0 cloudbees-bitbucket-branch-source:791.vb_eea_a_476405b cloudbees-disk-usage-simple:178.v1a_4d2f6359a_8 cloudbees-folder:6.758.vfd75d09eea_a_1 command-launcher:1.2 commons-lang3-api:3.12.0-36.vd97de6465d5b_ commons-text-api:1.10.0-27.vb_fa_3896786a_7 configuration-as-code:1569.vb_72405b_80249 copyartifact:1.47 credentials:1189.vf61b_a_5e2f62e credentials-binding:523.vd859a_4b_122e6 data-tables-api:1.12.1-4 display-url-api:2.3.6 docker-commons:1.21 docker-workflow:528.v7c193a_0b_e67c dtkit-api:3.0.2 durable-task:501.ve5d4fc08b0be echarts-api:5.4.0-1 email-ext:2.92 extended-read-permission:3.2 favorite:2.4.1 font-awesome-api:6.2.0-3 git:4.13.0 git-client:3.13.0 github:1.36.0 github-api:1.303-400.v35c2d8258028 github-branch-source:1696.v3a_7603564d04 groovy-postbuild:2.5 handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953 hashicorp-vault-plugin:359.v2da_3b_45f17d5 htmlpublisher:1.31 http_request:1.16 instance-identity:116.vf8f487400980 ionicons-api:31.v4757b_6987003 jackson2-api:2.13.4.20221013-295.v8e29ea_354141 jakarta-activation-api:2.0.1-2 jakarta-mail-api:2.0.1-2 javax-activation-api:1.2.0-5 javax-mail-api:1.6.2-5 jaxb:2.3.7-1 jdk-tool:1.0 jenkins-design-language:1.25.8 jjwt-api:0.11.5-77.v646c772fddb_0 job-dsl:1.81 jobConfigHistory:1176.v1b_4290db_41a_5 jquery-detached:1.2.1 jquery3-api:3.6.1-2 jsch:0.1.55.61.va_e9ee26616e7 junit:1160.vf1f01a_a_ea_b_7f kubernetes:3734.v562b_b_a_627ea_c kubernetes-cli:1.10.3 kubernetes-client-api:5.12.2-193.v26a_6078f65a_9 kubernetes-credentials:0.9.0 ldap:2.12 locale:204.v2a_f305fe7e9d lockable-resources:1069.v726298f53f8c mailer:438.v02c7f0a_12fa_4 matrix-auth:2.6.11 matrix-project:785.v06b_7f47b_c631 metrics:4.2.10-405.v60a_9cc74e923 momentjs:1.1.1 monitoring:1.91.0 okhttp-api:4.9.3-108.v0feda04578cf opentelemetry:2.10.0 permissive-script-security:0.7 pipeline-build-step:2.18 pipeline-graph-analysis:195.v5812d95a_a_2f9 pipeline-groovy-lib:621.vb_44ce045b_582 pipeline-input-step:456.vd8a_957db_5b_e9 pipeline-milestone-step:101.vd572fef9d926 pipeline-model-api:2.2118.v31fd5b_9944b_5 pipeline-model-definition:2.2118.v31fd5b_9944b_5 pipeline-model-extensions:2.2118.v31fd5b_9944b_5 pipeline-rest-api:2.27 pipeline-stage-step:296.v5f6908f017a_5 pipeline-stage-tags-metadata:2.2118.v31fd5b_9944b_5 pipeline-stage-view:2.27 pipeline-utility-steps:2.14.0 plain-credentials:139.ved2b_9cf7587b plugin-util-api:2.18.0 popper2-api:2.11.6-2 prometheus:2.0.10 pubsub-light:1.17 robot:3.3.0 role-strategy:3.2.0 scm-api:621.vda_a_b_055e58f7 script-security:1218.v39ca_7f7ed0a_c simple-theme-plugin:136.v23a_15f86c53d snakeyaml-api:1.33-90.v80dcb_3814d35 sse-gateway:1.26 ssh-agent:295.v9ca_a_1c7cc3a_a_ ssh-credentials:305.v8f4381501156 ssh-slaves:2.854.v7fd446b_337c9 sshd:3.236.ved5e1b_cb_50b_2 structs:324.va_f5d6774f3a_d timestamper:1.21 token-macro:321.vd7cc1f2a_52c8 trilead-api:1.67.vc3938a_35172f variant:59.vf075fe829ccb windows-slaves:1.8.1 workflow-aggregator:590.v6a_d052e5a_a_b_5 workflow-api:1200.v8005c684b_a_c6 workflow-basic-steps:994.vd57e3ca_46d24 workflow-cps:3536.vb_8a_6628079d5 workflow-durable-task-step:1217.v38306d8fa_b_5c workflow-job:1254.v3f64639b_11dd workflow-multibranch:716.vc692a_e52371b_ workflow-scm-step:400.v6b_89a_1317c9a_ workflow-step-api:639.v6eca_cd8c04a_a_ workflow-support:839.v35e2736cfd5c xunit:3.1.2 ```What Operating System are you using (both controller, and any agents involved in the problem)?
Linux (Pipeline runs on master)
Reproduction steps
Expected Results
Pipeline prints
Actual Results
NullPointerException, because Stapler seems not available when calling directly from pipeline context (See) Similiar code works for e.g. BlueOcean, because there Stapler is available (See)
Anything else?
The shown code snippet is used to collect the output of all steps of a previous executed stage from within the same pipeline. The output is then parsed for e.g. warnings. I am not use BlueOcean REST API for that as this fails in some cases to retrieve the full stage log. Beside redirecting all step output to a stage.log file, I'm not aware of any other method to get stage console log from a running pipeline.