search

SumoLogic / sumologic-jenkins-plugin

Apache License 2.0
11 stars 19 forks source link

Jenkins instance is missing #49

Closed DuMaM closed 1 year ago

DuMaM commented 2 years ago

Could somebody help me fix this error.

Stack-trace
java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
      at jenkins.model.Jenkins.get([Jenkins.java:808](https://someJenkins.com/monitoring?part=source&class=jenkins.model.Jenkins#808))
      at com.sumologic.jenkins.jenkinssumologicplugin.PluginDescriptorImpl$PluginConfigurationFromMain.call([PluginDescriptorImpl.java:325](https://someJenkins.com/monitoring?part=source&class=com.sumologic.jenkins.jenkinssumologicplugin.PluginDescriptorImpl$PluginConfigurationFromMain#325))
      at com.sumologic.jenkins.jenkinssumologicplugin.PluginDescriptorImpl$PluginConfigurationFromMain.call([PluginDescriptorImpl.java:320](https://someJenkins.com/monitoring?part=source&class=com.sumologic.jenkins.jenkinssumologicplugin.PluginDescriptorImpl$PluginConfigurationFromMain#320))
      at hudson.remoting.UserRequest.perform([UserRequest.java:211](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.UserRequest#211))
      at hudson.remoting.UserRequest.perform([UserRequest.java:54](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.UserRequest#54))
      at hudson.remoting.Request$2.run([Request.java:376](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.Request$2#376))
      at hudson.remoting.InterceptingExecutorService.lambda$wrap$0([InterceptingExecutorService.java:78](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.InterceptingExecutorService#78))
      at java.util.concurrent.FutureTask.run([FutureTask.java:264](https://someJenkins.com/monitoring?part=source&class=java.util.concurrent.FutureTask#264))
      at java.util.concurrent.ThreadPoolExecutor.runWorker([ThreadPoolExecutor.java:1128](https://someJenkins.com/monitoring?part=source&class=java.util.concurrent.ThreadPoolExecutor#1128))
      at java.util.concurrent.ThreadPoolExecutor$Worker.run([ThreadPoolExecutor.java:628](https://someJenkins.com/monitoring?part=source&class=java.util.concurrent.ThreadPoolExecutor$Worker#628))
      at java.lang.Thread.run([Thread.java:834](https://someJenkins.com/monitoring?part=source&class=java.lang.Thread#834))
      Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to Channel to Maven [java, -cp, /home/hudson/build/maven3-agent.jar:/opt/jenkins/apache-maven-3.0.5/boot/plexus-classworlds-2.4.jar, org.jvnet.hudson.maven3.agent.Maven3Main, /opt/jenkins/apache-maven-3.0.5, /tmp/remoting.jar, /home/hudson/build/maven3-interceptor.jar, /home/hudson/build/maven3-interceptor-commons.jar, 44881]
            at hudson.remoting.Channel.attachCallSiteStackTrace([Channel.java:1784](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.Channel#1784))
            at hudson.remoting.UserRequest$ExceptionResponse.retrieve([UserRequest.java:356](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.UserRequest$ExceptionResponse#356))
            at hudson.remoting.Channel.call([Channel.java:1000](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.Channel#1000))
            at com.sumologic.jenkins.jenkinssumologicplugin.PluginDescriptorImpl.getPluginConfiguration([PluginDescriptorImpl.java:96](https://someJenkins.com/monitoring?part=source&class=com.sumologic.jenkins.jenkinssumologicplugin.PluginDescriptorImpl#96))
            at com.sumologic.jenkins.jenkinssumologicplugin.sender.LogSender.sendLogs([LogSender.java:102](https://someJenkins.com/monitoring?part=source&class=com.sumologic.jenkins.jenkinssumologicplugin.sender.LogSender#102))
            at com.sumologic.jenkins.jenkinssumologicplugin.sender.LogSender.sendLogs([LogSender.java:134](https://someJenkins.com/monitoring?part=source&class=com.sumologic.jenkins.jenkinssumologicplugin.sender.LogSender#134))
            at com.sumologic.jenkins.jenkinssumologicplugin.sender.LogSenderHelper.sendAuditLogs([LogSenderHelper.java:103](https://someJenkins.com/monitoring?part=source&class=com.sumologic.jenkins.jenkinssumologicplugin.sender.LogSenderHelper#103))
            at com.sumologic.jenkins.jenkinssumologicplugin.utility.CommonModelFactory.captureAuditEvent([CommonModelFactory.java:363](https://someJenkins.com/monitoring?part=source&class=com.sumologic.jenkins.jenkinssumologicplugin.utility.CommonModelFactory#363))
            at com.sumologic.jenkins.jenkinssumologicplugin.listeners.SumoPipelineStatusListener.onCompleted([SumoPipelineStatusListener.java:88](https://someJenkins.com/monitoring?part=source&class=com.sumologic.jenkins.jenkinssumologicplugin.listeners.SumoPipelineStatusListener#88))
            at hudson.model.listeners.RunListener.lambda$fireCompleted$0([RunListener.java:207](https://someJenkins.com/monitoring?part=source&class=hudson.model.listeners.RunListener#207))
            at jenkins.util.Listeners.lambda$notify$0([Listeners.java:59](https://someJenkins.com/monitoring?part=source&class=jenkins.util.Listeners#59))
            at jenkins.util.Listeners.notify([Listeners.java:67](https://someJenkins.com/monitoring?part=source&class=jenkins.util.Listeners#67))
            at hudson.model.listeners.RunListener.fireCompleted([RunListener.java:205](https://someJenkins.com/monitoring?part=source&class=hudson.model.listeners.RunListener#205))
            at hudson.maven.MavenBuild$ProxyImpl2.end([MavenBuild.java:664](https://someJenkins.com/monitoring?part=source&class=hudson.maven.MavenBuild$ProxyImpl2#664))
            at sun.reflect.GeneratedMethodAccessor2205.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke([DelegatingMethodAccessorImpl.java:43](https://someJenkins.com/monitoring?part=source&class=sun.reflect.DelegatingMethodAccessorImpl#43))
            at java.lang.reflect.Method.invoke([Method.java:498](https://someJenkins.com/monitoring?part=source&class=java.lang.reflect.Method#498))
            at hudson.model.Executor$2.call([Executor.java:962](https://someJenkins.com/monitoring?part=source&class=hudson.model.Executor$2#962))
            at hudson.util.InterceptingProxy$1.invoke([InterceptingProxy.java:24](https://someJenkins.com/monitoring?part=source&class=hudson.util.InterceptingProxy$1#24))
            at com.sun.proxy.$Proxy153.end(Unknown Source)
            at sun.reflect.GeneratedMethodAccessor2205.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke([DelegatingMethodAccessorImpl.java:43](https://someJenkins.com/monitoring?part=source&class=sun.reflect.DelegatingMethodAccessorImpl#43))
            at java.lang.reflect.Method.invoke([Method.java:498](https://someJenkins.com/monitoring?part=source&class=java.lang.reflect.Method#498))
            at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform([RemoteInvocationHandler.java:924](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.RemoteInvocationHandler$RPCRequest#924))
            at hudson.remoting.RemoteInvocationHandler$RPCRequest.call([RemoteInvocationHandler.java:902](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.RemoteInvocationHandler$RPCRequest#902))
            at hudson.remoting.RemoteInvocationHandler$RPCRequest.call([RemoteInvocationHandler.java:853](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.RemoteInvocationHandler$RPCRequest#853))
            at hudson.remoting.UserRequest.perform([UserRequest.java:211](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.UserRequest#211))
            at hudson.remoting.UserRequest.perform([UserRequest.java:54](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.UserRequest#54))
            at hudson.remoting.Request$2.run([Request.java:376](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.Request$2#376))
            at hudson.remoting.InterceptingExecutorService.lambda$wrap$0([InterceptingExecutorService.java:78](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.InterceptingExecutorService#78))
            at org.jenkinsci.remoting.CallableDecorator.call([CallableDecorator.java:18](https://someJenkins.com/monitoring?part=source&class=org.jenkinsci.remoting.CallableDecorator#18))
            at hudson.remoting.CallableDecoratorList.lambda$applyDecorator$0([CallableDecoratorList.java:19](https://someJenkins.com/monitoring?part=source&class=hudson.remoting.CallableDecoratorList#19))
            at jenkins.util.ContextResettingExecutorService$2.call([ContextResettingExecutorService.java:46](https://someJenkins.com/monitoring?part=source&class=jenkins.util.ContextResettingExecutorService$2#46))
            at jenkins.security.ImpersonatingExecutorService$2.call([ImpersonatingExecutorService.java:80](https://someJenkins.com/monitoring?part=source&class=jenkins.security.ImpersonatingExecutorService$2#80))
            at java.util.concurrent.FutureTask.run([FutureTask.java:266](https://someJenkins.com/monitoring?part=source&class=java.util.concurrent.FutureTask#266))
            at java.util.concurrent.ThreadPoolExecutor.runWorker([ThreadPoolExecutor.java:1142](https://someJenkins.com/monitoring?part=source&class=java.util.concurrent.ThreadPoolExecutor#1142))
            at java.util.concurrent.ThreadPoolExecutor$Worker.run([ThreadPoolExecutor.java:617](https://someJenkins.com/monitoring?part=source&class=java.util.concurrent.ThreadPoolExecutor$Worker#617))
            at java.lang.Thread.run([Thread.java:745](https://someJenkins.com/monitoring?part=source&class=java.lang.Thread#745))
sumo-drosiek commented 2 years ago

cc: @SumoLogic/sumoappdev

DuMaM commented 2 years ago

Any news here? 😉 It started to happening since 2.289 LTS, and now it's present on 2.332.3. It's happening in maven projects or in pipelines with Maven Pipeline Plugin. If you need any additional info, I can provide you anything you need.

sumoanema commented 2 years ago

@DuMaM - Can you share the Jenkins service log. what was the triggering point of the error. Did it cause due to installed/updated the plugin. Does it occur as soon as you restart the jenkins ?

sumoanema commented 2 years ago

@DuMaM - Any update on this ?

DuMaM commented 2 years ago

Hi @sumoanema , I was out of office and wasnt able to provide you any details. sorry for delay 🙏

I will try to answer your questions:

DuMaM commented 2 years ago

@sumoanema Maven version/JDK version do not affect on behaviour. Do you need something more?

sumoanema commented 2 years ago

sorry for the delay...will try to analyse the above information and get back ASAP.

DuMaM commented 2 years ago

Sure I also reported this in jenkins for maven plugin, becasue it was spotted there at first. https://issues.jenkins.io/browse/JENKINS-69755

sumoanema commented 2 years ago

@DuMaM - Have you tried running the job without sumo logic plugin..does it still fail ? Are any type log getting ingested in sumo through the plugin are no logs are getting sent ? Does your pipeline have only maven console logs or are there is anything else ?

Also I will advice you to raise a support ticket with sumologic to track this better. You can answer the above question. I think we might need to get on a call for this .

DuMaM commented 2 years ago

Sure, i will tests this in Monday.

DuMaM commented 2 years ago

@sumoanema is this doable? https://github.com/SumoLogic/sumologic-jenkins-plugin/issues/50#issue-1410115561

I'm wondering if you could upgrade this plugin first, maybe this issue will be solved just by bumping versions :)

DuMaM commented 2 years ago

Have you tried running the job without sumo logic plugin..does it still fail ?

Not yet but I will.

Are any type log getting ingested in sumo through the plugin are no logs are getting sent ?

Jenkins sent whole build log to sumo.

Does your pipeline have only maven console logs or are there is anything else ?

As I wrote before it's a Maven Projects, not FreeStyle, not Pipeline one. This means -> Yes 😉 https://plugins.jenkins.io/maven-plugin/

Also I will advice you to raise a support ticket with sumologic to track this better. You can answer the above question. I think we might need to get on a call for this .

Sure. I will link this issue in it.

sumoanema commented 2 years ago

@DuMaM - Just to clarify this - "Jenkins sent whole build log to sumo." When the job tries to send logs to sumo are there any logs which are received at sumo end. Or there are no logs received.

DuMaM commented 2 years ago

Have you tried running the job without sumo logic plugin..does it still fail ?

I just tried. No fails since 3h, so the sumo plugin for sure is source of this issue.

DuMaM commented 2 years ago

Just to clarify this - "Jenkins sent whole build log to sumo." When the job tries to send logs to sumo are there any logs which are received at sumo end. Or there are no logs received.

Not every job report all logs. It's hard to determine why this is happening, but some jobs have full log and some not.

sumoanema commented 2 years ago

@DuMaM - I think we will need to get on a call for this. Since the error message is quite confusing. The thing is if the jenkins.instance is missing, ideally it should not have been able to send any logs. But based on your last comment it seems like your are able to get some logs and the jenkins plugin somehow fails due to some problem in between.

I hope you have raised a support ticket with sumo ?

DuMaM commented 2 years ago

Ticket number #76941

DuMaM commented 2 years ago

Small update In sumo ticket we agreed to update this plugin according to https://github.com/SumoLogic/sumologic-jenkins-plugin/issues/50

DuMaM commented 2 years ago

@sumo-drosiek @sumoanema Any updates?

sumoanema commented 2 years ago

Hi @DuMaM - I am trying to update the jenkins plugin version but getting few build issues currently. Another thing which I wanted to try was to replicate the exact error. For which I might need the exact version of your set up for java,maven version which you are using. And as you said earlier jenkins version used by you have been : 2.289 LTS and 2.332.3

I had asked the same info over sumo support ticket and was awaiting your response there.

DuMaM commented 2 years ago

@sumoanema https://issues.jenkins.io/browse/JENKINS-69755 here you have env settings

sumoanema commented 2 years ago

@DuMaM - Thanks let me try to replicate the exact same issue and get back ASAP

sumoanema commented 1 year ago

update : @DuMaM - I was able to reproduce the issue in my local setup. On the other hand when trying to upgrade the jenkins plugin I am running into this error : https://issues.jenkins.io/browse/JENKINS-54844?page=com.atlassian.jira.plugin.system.issuetabpanels%3Achangehistory-tabpanel though the issue is closed .

I have tried upgrading it to 2.361.1, 2.289, 2.332.3 but all lead into the same error. I am currently looking into this and trying to find any solution (maybe upgrade plugin or some other code fix)

sumoanema commented 1 year ago

@DuMaM - In my setup all the logs are getting ingested in sumo from the maven job...its just that we keep getting this single error message (jenkins.instance missing) send to sumo. Also the jenkins log also have the stack trace.

But hopefully the maven job logs are still getting ingested for you as well to work with ?

DuMaM commented 1 year ago

its just that we keep getting this single error message (jenkins.instance missing) send to sumo

I'm also getting this, and I got 2 million such errors per day... This makes tracking other issues a problematic.

In my setup all the logs are getting ingested in sumo from the maven job, but hopefully the maven job logs are still getting ingested for you as well to work with ?

Maybe there is also other issue that leads to such behavior. 🤷 For us some of them are lost and some not. After you fix this jenkins.instance is missing, I will create new ticket if I find something new.

DuMaM commented 1 year ago

@sumoanema any updates?

DuMaM commented 1 year ago

Bump?

sumoanema commented 1 year ago

@DuMaM - Sorry for the late reply. Just got back from vacation. I am still working on it. Will try to provide a fix as soon as possible.

DuMaM commented 1 year ago

@sumoanema take your time 😉

DuMaM commented 1 year ago

@sumoanema do you have any ETA on this issue?

sumoanema commented 1 year ago

@DuMaM - let me get back with the ETA in a couple of days.

DuMaM commented 1 year ago

I got experience in plugins modernization. Maybe you can make your PR public so we all can contribute to it and finish your work faster?

sumoanema commented 1 year ago

There are some security vulnerability which are reported in the plugin for which I upgraded some jar and realised while building that jenkins.version also needs to in upgraded, which might solve your issue and vulnerability issue. But I am still stuck with build issue. https://issues.jenkins.io/browse/JENKINS-54844?page=com.atlassian.jira.plugin.system.issuetabpanels%3Achangehistory-tabpanel

Here is the PR with the change : https://github.com/SumoLogic/sumologic-jenkins-plugin/pull/53

DuMaM commented 1 year ago

Any updates?

sumoanema commented 1 year ago

@DuMaM - Still stuck with the same error - https://issues.jenkins.io/browse/JENKINS-54844 . Trying to find a way out. Tried your suggestion of using bom from - https://github.com/SumoLogic/sumologic-jenkins-plugin/pull/53 But the thing is the error does not seem to be a dependency problem.

Also as suggested according to install state plugin can be migrated to at least 2.332.1. I did the same. When building with java 8 everyhting goes fine. But as soon as I upgrade to java 11 (and resolving all the build issue) I finally get stuck with the same problem.

DuMaM commented 1 year ago

Let's do baby steps. Maybe upgrade plugin first and then let's search for source of an issue

DuMaM commented 1 year ago

@sumoanema when this issue will be solved? any eta?

sumoanema commented 1 year ago

@DuMaM - as mentioned in issue #50 I have upgraded the plugin to 2.332.1 (with java 8) but I have been experiencing some problem in publishing the same to jenkins artifactory. Also at least for my environment that does not seem to be solving the underlying issue of "jenkins.instance" missing.

As far as the ETA is concern I am in talks with my team to get this prioritised. I will try to get back with an ETA as soon as possible.

DuMaM commented 1 year ago

@DuMaM - as mentioned in issue https://github.com/SumoLogic/sumologic-jenkins-plugin/issues/50 I have upgraded the plugin to 2.332.1 (with java 8) but I have been experiencing some problem in publishing the same to jenkins artifactory. Also at least for my environment that does not seem to be solving the underlying issue of "jenkins.instance" missing.

It's ok, as I said, let's do this with baby steps, and upgrade plugin first. I'm also a plugin maintainer, here is how I set up my project for app releases. Maybe it will be useful for you. https://github.com/jenkinsci/crowd2-plugin/blob/master/_init.sh

How to perform release you can find in plugin documents. https://www.jenkins.io/doc/developer/publishing/releasing-manually/

If you can't log in into Artifactory, please create a ticket here: https://github.com/jenkins-infra/helpdesk

You can also ask for help in Jenkins Google Group, they are really helpful :) https://groups.google.com/g/jenkinsci-dev

As far as the ETA is concern I am in talks with my team to get this prioritised. I will try to get back with an ETA as soon as possible.

Sure, I will reach you in a week or two.

DuMaM commented 1 year ago

@sumoanema any updates?

sumoanema commented 1 year ago

@DuMaM have been facing some build issue while releasing the plugin. Have been looking into the same. Will keep this ticket posted with the updates.

DuMaM commented 1 year ago

@sumoanema please use Jenkins BOM for plugins, it will help a lot. There is also a modernization video on Jenkins YouTube site where all steps are explained how to proceed with plugin build process.

sumoanema commented 1 year ago

@DuMaM - I was able to upgrade the jenkins plugin with java 11 and jenkins.version to 2.361.1, and was able to compile the plugin successfully. Upgrade the parent pom version to 4.55 for the plugin. Currently I am facing an issue with in build test cases (which is using mockito). _Wanted but not invoked: httpRequestHandler.handle(

, isA(org.apache.http.HttpResponse), isA(org.apache.http.protocol.HttpContext) ); -> at com.sumologic.jenkins.jenkinssumologicplugin.sender.SumoPeriodicPublisherTest.execute(SumoPeriodicPublisherTest.java:37) Actually, there were zero interactions with this mock._ Although I have not made any code change and have done just the biuld dependency upgrades, I am not sure why I am facing this issue. I am currently looking into it.
DuMaM commented 1 year ago

To be fair we need to remove a sumologic plugin and drop the deal It do not work in last 2.375.4 LTS

sumoanema commented 1 year ago

@DuMaM - Sorry I missed the your last comment. I did not get you there. What do you mean by "remove a sumologic plugin and drop the deal" ? Do you mean - your jenkins system is upgraded to 2.375.4 and sumo plugin is not working with the same ? Hence you have stopped using it ?

Also latest update from my side - I was able to build the plugin sucessfully with jenkins.version 2.361.4. But the underlying error of "jenkins.instance is missing" is not resolved by this upgrade. I am using java 11 to run my jenkins instance. I am having a look into it.

sumoanema commented 1 year ago

@DuMaM - Just wanted to know about your last comment. Do you want the jenkins plugin's jenkins.version to be available on 2.375.4 ?

DuMaM commented 1 year ago

It took us a while to fix it. After merging from 2.361.3 to 2.375.4 something broke in plugin. It wasn't able to load itself correctly. Multiple resets didn't help. Finally we removed a plugin with configs, and then installed it once again. And after this operation it started to work correctly.

This means something in the serialization of classes was changed, and Jenkins wasn't able to parse stored config on xmls.

sumoanema commented 1 year ago

This repository is getting archived and the plugin will be maintained in jenkins repo itself - https://github.com/jenkinsci/sumologic-publisher-plugin as suggested by Jenkins open source team in this ticket - https://github.com/jenkins-infra/helpdesk/issues/3670 . Closing this ticket before archiving this and will track this as a bug internally.