[JENKINS-18687] Intermittent failure whilst recording test results caused by java.util.MissingResourceException

[timja]

I've recently added a few extra slaves to my Jenkins CI environment, and I'm occasionally getting an otherwise successful build failing whilst "Recording test results".

I've included the full stack trace below, but it seems to relate to a missing resource bundle by com.sun.org.apache.xerces.internal.impl.msg.SAXMessages.

What's strange is that this only happens intermittently, and I've yet to find a conclusive pattern to the failures.

BUILD SUCCESSFUL

Total time: 2 mins 35.991 secs
Archiving artifacts
Recording test results
ERROR: Failed to archive test reports
hudson.util.IOException2: remote file operation failed: /home/ci/jenkins/workspace/experimental-tim-unit at hudson.remoting.Channel@7f5f62bf:st-jenkinsslave-003
    at hudson.FilePath.act(FilePath.java:900)
    at hudson.FilePath.act(FilePath.java:877)
    at hudson.tasks.junit.JUnitParser.parse(JUnitParser.java:87)
    at hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:121)
    at hudson.tasks.junit.JUnitResultArchiver.perform(JUnitResultArchiver.java:133)
    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:802)
    at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:774)
    at hudson.model.Build$BuildExecution.post2(Build.java:183)
    at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:724)
    at hudson.model.Run.execute(Run.java:1600)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:237)
Caused by: hudson.util.IOException2: Failed to read /home/ci/jenkins/workspace/experimental-tim-unit/TIM/DBUpgrade/build/test-results/TEST-dbupgrade.framework.DBUpgradeSQLTest.xml
    at hudson.tasks.junit.TestResult.parse(TestResult.java:284)
    at hudson.tasks.junit.TestResult.parsePossiblyEmpty(TestResult.java:223)
    at hudson.tasks.junit.TestResult.parse(TestResult.java:158)
    at hudson.tasks.junit.TestResult.parse(TestResult.java:141)
    at hudson.tasks.junit.TestResult.(TestResult.java:117)
    at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:117)
    at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:90)
    at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2387)
    at hudson.remoting.UserRequest.perform(UserRequest.java:118)
    at hudson.remoting.UserRequest.perform(UserRequest.java:48)
    at hudson.remoting.Request$2.run(Request.java:326)
    at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:679)
Caused by: org.dom4j.DocumentException: Could not load any resource bundle by com.sun.org.apache.xerces.internal.impl.msg.SAXMessages Nested exception: Could not load any resource bundle by com.sun.org.apache.xerces.internal.impl.msg.SAXMessages
    at org.dom4j.io.SAXReader.read(SAXReader.java:484)
    at org.dom4j.io.SAXReader.read(SAXReader.java:264)
    at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:129)
    at hudson.tasks.junit.TestResult.parse(TestResult.java:267)
    ... 16 more
Caused by: java.util.MissingResourceException: Could not load any resource bundle by com.sun.org.apache.xerces.internal.impl.msg.SAXMessages
    at com.sun.org.apache.xerces.internal.utils.SecuritySupport$7.run(SecuritySupport.java:172)
    at com.sun.org.apache.xerces.internal.utils.SecuritySupport$7.run(SecuritySupport.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at com.sun.org.apache.xerces.internal.utils.SecuritySupport.getResourceBundle(SecuritySupport.java:164)
    at com.sun.org.apache.xerces.internal.util.SAXMessageFormatter.formatMessage(SAXMessageFormatter.java:57)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.getProperty(AbstractSAXParser.java:2063)
    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.setProperty(SAXParserImpl.java:470)
    at org.dom4j.io.SAXHelper.setParserProperty(SAXHelper.java:34)
    at org.dom4j.io.SAXReader.configureReader(SAXReader.java:911)
    at org.dom4j.io.SAXReader.read(SAXReader.java:463)
    ... 19 more
Build step 'Publish JUnit test result report' changed build result to FAILURE
Notifying upstream projects of job completion
Finished: FAILURE

---

Originally reported by scarytom, imported from: Intermittent failure whilst recording test results caused by java.util.MissingResourceException

• status: Open
• priority: Major
• resolution: Unresolved
• imported: 2022/01/10

[timja]

plaa:

I have the same issue (after adding a new slave), but it doesn't seem intermittent. I've received it twice in a row.

The console output just indicates a failure, though the XML file does not contain any errors:

Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Finished: UNSTABLE

When I go check the test results, I find the following stack trace:

Failed to read test report file /home/weller/jenkins/workspace/TEST_VISUAL/output/junit/TEST-features-visuals-visuals.xml
org.dom4j.DocumentException: Could not load any resource bundle by com.sun.org.apache.xerces.internal.impl.msg.SAXMessages Nested exception: Could not load any resource bundle by com.sun.org.apache.xerces.internal.impl.msg.SAXMessages
at org.dom4j.io.SAXReader.read(SAXReader.java:484)
at org.dom4j.io.SAXReader.read(SAXReader.java:264)
at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:130)
at hudson.tasks.junit.TestResult.parse(TestResult.java:274)
at hudson.tasks.junit.TestResult.parsePossiblyEmpty(TestResult.java:230)
at hudson.tasks.junit.TestResult.parse(TestResult.java:165)
at hudson.tasks.junit.TestResult.parse(TestResult.java:148)
at hudson.tasks.junit.TestResult.(TestResult.java:124)
at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:117)
at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:90)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2439)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:328)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)
Caused by: java.util.MissingResourceException: Could not load any resource bundle by com.sun.org.apache.xerces.internal.impl.msg.SAXMessages
at com.sun.org.apache.xerces.internal.utils.SecuritySupport$7.run(SecuritySupport.java:174)
at com.sun.org.apache.xerces.internal.utils.SecuritySupport$7.run(SecuritySupport.java:166)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.org.apache.xerces.internal.utils.SecuritySupport.getResourceBundle(SecuritySupport.java:166)
at com.sun.org.apache.xerces.internal.util.SAXMessageFormatter.formatMessage(SAXMessageFormatter.java:57)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.getProperty(AbstractSAXParser.java:2060)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.setProperty(SAXParserImpl.java:490)
at org.dom4j.io.SAXHelper.setParserProperty(SAXHelper.java:34)
at org.dom4j.io.SAXReader.configureReader(SAXReader.java:911)
at org.dom4j.io.SAXReader.read(SAXReader.java:463)
... 19 more
Nested exception:
java.util.MissingResourceException: Could not load any resource bundle by com.sun.org.apache.xerces.internal.impl.msg.SAXMessages
at com.sun.org.apache.xerces.internal.utils.SecuritySupport$7.run(SecuritySupport.java:174)
at com.sun.org.apache.xerces.internal.utils.SecuritySupport$7.run(SecuritySupport.java:166)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.org.apache.xerces.internal.utils.SecuritySupport.getResourceBundle(SecuritySupport.java:166)
at com.sun.org.apache.xerces.internal.util.SAXMessageFormatter.formatMessage(SAXMessageFormatter.java:57)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.getProperty(AbstractSAXParser.java:2060)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.setProperty(SAXParserImpl.java:490)
at org.dom4j.io.SAXHelper.setParserProperty(SAXHelper.java:34)
at org.dom4j.io.SAXReader.configureReader(SAXReader.java:911)
at org.dom4j.io.SAXReader.read(SAXReader.java:463)
at org.dom4j.io.SAXReader.read(SAXReader.java:264)
at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:130)
at hudson.tasks.junit.TestResult.parse(TestResult.java:274)
at hudson.tasks.junit.TestResult.parsePossiblyEmpty(TestResult.java:230)
at hudson.tasks.junit.TestResult.parse(TestResult.java:165)
at hudson.tasks.junit.TestResult.parse(TestResult.java:148)
at hudson.tasks.junit.TestResult.(TestResult.java:124)
at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:117)
at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:90)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2439)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:328)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)

[timja]

plaa:

The problem disappeared when I updated all my Jenkins plugins. No idea which one fixed the problem. And no idea why my other slave was working without problems.

[timja]

lvotypkova:

It happens on our instance when java which is in java.home of slave.jar is updated. Do you remember if there was some java updates during slave connection in jenkins?

[timja]

tomc:

I've also just experienced this issue, and it does appear to be related to an update of java. Restarting the container (Tomcat, in my case) and executing the job again resolved the issue.

[timja]

tjoneslo:

I am now/also seeing this same issue on my Jenkins, but consistently. Every time I run a job which invokes the JUnit plugin I get a failure:

hudson.remoting.Channel@20f57bec:: java.io.IOException: Remote call on failed
Caused by: java.io.IOException: Remote call on failed
Caused by: java.lang.ExceptionInInitializerError
Caused by: java.util.MissingResourceException: Can't find bundle for base name javax.servlet.LocalStrings, locale en_US

I'm running Jenkins 1.598.

I've tried the suggestions about [up|down]grading the JUnit plugin. I've tried 1.1, 1.2beta4, 1.3, and 1.4 all with the same failure. I restarted Jenkins between each trial. I've verified that both the master and slave machines have at least a minimal tomcat servlet api installation with a jar file which supplies the LocalStrings.properties file.

I'm pretty sure that the JUnit plugin is missing a key dependency. But one of the other plugin's supplies it for most other instances. I dread going through all the other plugins to find the one I need.

[timja]

tjoneslo:

New datapoint. In our Jenkins (v 1.598) configuration we have two Maven installations. First is Maven 3.0.5, the second is Maven 3.2.2. Both configured to install from Apache. I have one maven Job which does a "mvn -U clean install". When configured for the 3.0.5 version of maven, the JUnit gathering test results fails with the MissingResourceException. If I change the job to use Maven 3.2.2, the JUnit gathering results succeeds and the Job passes.

[timja]

yurigo79:

I was able to solve this by deleting the ~/.jenkins folder on the problematic slave

[timja]

olivergondza:

What Java version was used when this exception was thrown?

[timja]

yurigo79:

olivergondza - in my case it was

java version "1.6.0_41"
Java(TM) SE Runtime Environment (build 1.6.0_41-b02)
Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01, mixed mode)

[timja]

lvotypkova:

I did a more investigation - I finally got the problem when it occurs - usually slave was restarted and fine when I received the ticket about this problem.
I even find exact place where it is. To more explanation there is a difference in obtaining property.

for example my code on affected slave:

PropertyResourceBundle.getBundle("com.sun.org.apache.xerces.internal.impl.msg.SAXMessages", Locale.US)

this return right bundle without problems.

but calling (as it is done in by in SAXReader used by this plugin)

SecuritySupport.getResourceBundle("com.sun.org.apache.xerces.internal.impl.msg.SAXMessages", new Locale("en", "US"));

this throws exception MissingResourceException.

It is very strange because SecuritySupport.getResourceBundle() calls the same code (PropertyResourceBundle.getBundle("com.sun.org.apache.xerces.internal.impl.msg.SAXMessages", Locale.US) ) bud under AccessController.doPrivileged():

http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/com/sun/org/apache/xerces/internal/utils/SecuritySupport.java#165

It seems that there is some difference in execution this code (despite it is the same). I will try to investigate what is different. That could be a way how to realized what is wrong. I think that it maybe will not be problem in plugin itself (bug in jdk or remoting or leaking some classes for master....).
Have any of us idea?

[timja]

saki:

I was experiencing failure caused by the message "Could not load any resource bundle by com.sun.org.apache.xerces.internal.impl.msg.XMLMessages" when was upgraded java (by yum update) while application server (in my case tomcat) was still running. The restart of server resolved the issue.

[timja]

[Originally duplicated by: JENKINS-22145]

[timja]

[Originally duplicated by: JENKINS-27415]