Closed xhernandez closed 1 year ago
Seems a automated update of jenkins happened. I restarted it, no luck. I am investigating.
So it fail with this error as seen with curl:
HTTP ERROR 500 javax.servlet.ServletException: org.apache.commons.jelly.JellyTagException: jar:file:/var/lib/jenkins/%25C/jenkins/war/WEB-INF/lib/jenkins-core-2.387.1.jar!/jenkins/views/Jenkins* TLSv1.2 (IN), TLS header, Supplemental data (23):
Header/headerContent.jelly:71:88: <st:include> org/apache/commons/httpclient/URIException
URI: | / |
---|---|
STATUS: | 500 |
MESSAGE: | javax.servlet.ServletException: org.apache.commons.jelly.JellyTagException: jar:file:/var/lib/jenkins/%25C/jenkins/war/WEB-INF/lib/jenkins-core-2.387.1.jar!/jenkins/views/JenkinsHeader/headerContent.jelly:71:88: <st:include> org/apache/commons/httpclient/URIException |
SERVLET: | Stapler |
CAUSED BY: | javax.servlet.ServletException: org.apache.commons.jelly.JellyTagException: jar:file:/var/lib/jenkins/%25C/jenkins/war/WEB-INF/lib/jenkins-core-2.387.1.jar!/jenkins/views/JenkinsHeader/headerContent.jelly:71:88: <st:include> org/apache/commons/httpclient/URIException |
CAUSED BY: | org.apache.commons.jelly.JellyTagException: jar:file:/var/lib/jenkins/%25C/jenkins/war/WEB-INF/lib/jenkins-core-2.387.1.jar!/jenkins/views/JenkinsHeader/headerContent.jelly:71:88: <st:include> org/apache/commons/httpclient/URIException |
CAUSED BY: | java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException |
javax.servlet.ServletException: org.apache.commons.jelly.JellyTagException: jar:file:/var/lib/jenkins/%25C/jenkins/war/WEB-INF/lib/jenkins-core-2.387.1.jar!/jenkins/views/JenkinsHeader/headerContent.jelly:71:88: <st:include> org/apache/commons/httpclient/URIException at org.kohsuke.stapler.Facet$1.dispatch(Facet.java:249) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:770) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:900) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:698) at hudson.init.impl.InstallUncaughtExceptionHandler.handleException(InstallUncaughtExceptionHandler.java:59) at hudson.init.impl.InstallUncaughtExceptionHandler.lambda$init$0(InstallUncaughtExceptionHandler.java:33) at org.kohsuke.stapler.compression.CompressionFilter.repo* TLSv1.2 (IN), TLS header, Supplemental data (23): rtException(CompressionFilter.java:72) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:56) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:86) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) at jenkins.security.SuspiciousRequestFilter.doFilter(SuspiciousRequestFilter.java:38) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:202) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:549) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1570) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1383) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1543) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1305) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) at org.eclipse.jetty.server.Server.handle(Server.java:563) at org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:416) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:385) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:272) at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.lambda$new$0(AdaptiveExecutionStrategy.java:140) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:934) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1078) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: org.apache.commons.jelly.JellyTagException: jar:file:/var/lib/jenkins/%25C/jenkins/war/WEB-INF/lib/jenkins-core-2.387.1.jar!/jenkins/views/JenkinsHeader/headerContent.jelly:71:88: <st:include> org/apache/commons/httpclient/URIException at org.apache.commons.jelly.impl.TagScript.handleException(TagScript.java:738) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:285) at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161) at org.apache.commons.jelly.tags.core.OtherwiseTag.doTag(OtherwiseTag.java:41) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:265) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161) at org.apache.commons.jelly.tags.core.ChooseTag.doTag(ChooseTag.java:38) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:265) at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:101) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:101) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:99) at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:172) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:265) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:122) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:101) * TLSv1.2 (IN), TLS header, Supplemental data (23): at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:101) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:101) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:265) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:122) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:122) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:99) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:66) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:55) at org.kohsuke.stapler.jelly.ScriptInvoker.execute(ScriptInvoker.java:56) at org.kohsuke.stapler.jelly.ScriptInvoker.execute(ScriptInvoker.java:43) at org.kohsuke.stapler.Facet$1.dispatch(Facet.java:240) ... 49 more Caused by: java.lang.NoClassDefFoundError: org/apache/commons/httpclient/URIException at java.base/java.lang.Class.getDeclaredMethods0(Native Method) at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166) at java.base/java.lang.Class.getDeclaredMethods(Class.java:2309) at org.kohsuke.stapler.ClassDescriptor.findMethods(ClassDescriptor.java:143) at org.kohsuke.stapler.ClassDescriptor.<init>(ClassDescriptor.java:80) at org.kohsuke.stapler.lang.KlassNavigator$1.getFunctions(KlassNavigator.java:217) at org.kohsuke.stapler.lang.KlassNavigator$1.getFunctions(KlassNavigator.java:142) * TLSv1.2 (IN), TLS header, Supplemental data (23): at org.kohsuke.stapler.lang.Klass.getFunctions(Klass.java:98) at org.kohsuke.stapler.KlassDescriptor.<init>(KlassDescriptor.java:27) at org.kohsuke.stapler.MetaClass.buildDispatchers(MetaClass.java:111) at org.kohsuke.stapler.MetaClass.<init>(MetaClass.java:99) at org.kohsuke.stapler.WebApp$1.computeValue(WebApp.java:230) at org.kohsuke.stapler.WebApp$1.computeValue(WebApp.java:227) at java.base/java.lang.ClassValue.getFromHashMap(ClassValue.java:228) at java.base/java.lang.ClassValue.getFromBackup(ClassValue.java:210) at java.base/java.lang.ClassValue.get(ClassValue.java:116) at org.kohsuke.stapler.WebApp.getMetaClass(WebApp.java:246) at org.kohsuke.stapler.WebApp.getMetaClass(WebApp.java:240) at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:134) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:265) ... 93 more
I am downgrading the jenkins RPM for the moment.
Ok so that's back. Now, I need to see what happened, and how to block the upgrade for now.
So I think one library was removed from jenkins:
https://github.com/jenkinsci/jenkins/pull/7312
And some plugins need to be updated. I will update them, try the update, and if not, I will revert the jenkins update until I find the plugin causing trouble.
So, while on it, I removed all plugins who were not used, but still installed (like the gerrit one...). I am updating everything, and it will restart again.
Then I will check why it take ages to find a builder.
Ok so I updated jenkins and plugins and now it seems to work fine. I am finishing the last ones.
However, if anything is broken, do not hesitate to ping us.
And closing. Jenkins is back, I updated everything possible, and there is no warning.
Thanks for looking into it, but it seems something is not working fine yet:
https://build.gluster.org/job/gh_devrpm-el7/3958/console
13:02:41 GitHub pull request #4032 of commit 540fe33f1cdfcea87593bd0245bad9919ed3dbcc, no merge conflicts.
13:02:41 Running as SYSTEM
13:02:47 Setting status of 540fe33f1cdfcea87593bd0245bad9919ed3dbcc to PENDING with url https://build.gluster.org/job/gh_devrpm-el7/3958/ and message: 'Running: devrpm-el7'
13:02:47 Using context: Testing: devrpm-el7
13:02:47 Building remotely on [builder-c7-4.int.aws.gluster.org](https://build.gluster.org/computer/builder-c7-4.int.aws.gluster.org) (asan centos7) in workspace /home/jenkins/root/workspace/gh_devrpm-el7
13:02:48 The recommended git tool is: NONE
13:02:55 No credentials specified
13:02:55 Wiping out workspace first.
13:02:56 Also: java.nio.file.FileSystemException: /home/jenkins/root/workspace/gh_devrpm-el7/RPMS/el7/x86_64/hw_info.log: Operation not permitted
13:02:56 at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:100)
13:02:56 at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
13:02:56 at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
13:02:56 at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setMode(UnixFileAttributeViews.java:254)
13:02:56 at java.base/sun.nio.fs.UnixFileAttributeViews$Posix.setPermissions(UnixFileAttributeViews.java:276)
[...]
13:03:07 jenkins.util.io.CompositeIOException: Unable to delete '/home/jenkins/root/workspace/gh_devrpm-el7'. Tried 3 times (of a maximum of 3) waiting 0.1 sec between attempts. (Discarded 25 additional exceptions)
13:03:07 at jenkins.util.io.PathRemover.forceRemoveDirectoryContents(PathRemover.java:87)
13:03:07 at hudson.Util.deleteContentsRecursive(Util.java:286)
13:03:07 at hudson.FilePath$DeleteContents.invoke(FilePath.java:1459)
13:03:07 at hudson.FilePath$DeleteContents.invoke(FilePath.java:1454)
13:03:07 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3578)
13:03:07 at hudson.remoting.UserRequest.perform(UserRequest.java:211)
[...]
Oh that's just a cleanup issue!
You can trigger again and check!
That's weird:
# ls -l /home/jenkins/root/workspace/gh_devrpm-el7/RPMS/el7/x86_64/hw_info.log
-rwxr-xr-x. 1 jenkins jenkins 1418 Mar 14 12:34 /home/jenkins/root/workspace/gh_devrpm-el7/RPMS/el7/x86_64/hw_info.log
Ok, I cleaned it manually, we will see how it goes.
It seems to be working now. Thanks !!!
Connecting to https://jenkins.gluster.org, returns a page with the Jenkins header but no content. It seems that something is broken in Jenkins.