search

LinuxForHealth / FHIR

The LinuxForHealth FHIR® Server and related projects
https://linuxforhealth.github.io/FHIR
Apache License 2.0
330 stars 157 forks source link

intermittent audit test failure in CI #4154

Open lmsurpre opened 1 year ago

lmsurpre commented 1 year ago

Describe the bug We were seeing intermittent audit test failures a while back. Those seemed to stabalize for a while, but recently we saw another failure for a test that usually passes.

From https://github.com/LinuxForHealth/FHIR/actions/runs/3819990229/jobs/6497976982

Error:  org.linuxforhealth.fhir.server.test.SearchRevIncludeTest.testCreateNutritionOrders  Time elapsed: 10.019 s  <<< FAILURE!
java.util.concurrent.ExecutionException: java.lang.AssertionError: expected [SUCCESSFUL] but found [SERVER_ERROR]
    at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
    at org.linuxforhealth.fhir.server.test.SearchRevIncludeTest.testCreateNutritionOrders(SearchRevIncludeTest.java:495)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
    at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:677)
    at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:221)
    at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
    at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:962)
    at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194)
    at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
    at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
    at org.testng.TestRunner.privateRun(TestRunner.java:806)
    at org.testng.TestRunner.run(TestRunner.java:601)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:433)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:427)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:387)
    at org.testng.SuiteRunner.run(SuiteRunner.java:330)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
    at org.testng.TestNG.runSuites(TestNG.java:1099)
    at org.testng.TestNG.run(TestNG.java:1067)
    at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:324)
    at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:74)
    at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:123)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
    at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
    at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)
Caused by: java.lang.AssertionError: expected [SUCCESSFUL] but found [SERVER_ERROR]
    at org.testng.Assert.fail(Assert.java:110)
    at org.testng.Assert.failNotEquals(Assert.java:1413)
    at org.testng.Assert.assertEqualsImpl(Assert.java:149)
    at org.testng.Assert.assertEquals(Assert.java:131)
    at org.testng.Assert.assertEquals(Assert.java:643)
    at org.linuxforhealth.fhir.server.test.SearchRevIncludeTest$NutritionOrderCallable.call(SearchRevIncludeTest.java:533)
    at org.linuxforhealth.fhir.server.test.SearchRevIncludeTest$NutritionOrderCallable.call(SearchRevIncludeTest.java:519)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

Error:  org.linuxforhealth.fhir.server.test.SearchRevIncludeTest.testDeleteNutritionOrders  Time elapsed: 1.016 s  <<< FAILURE!
java.util.concurrent.ExecutionException: java.lang.AssertionError: expected [false] but found [true]
    at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
    at org.linuxforhealth.fhir.server.test.SearchRevIncludeTest.testDeleteNutritionOrders(SearchRevIncludeTest.java:701)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
    at org.testng.internal.invokers.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:69)
    at org.testng.internal.invokers.ConfigInvoker.invokeConfigurationMethod(ConfigInvoker.java:361)
    at org.testng.internal.invokers.ConfigInvoker.invokeConfigurations(ConfigInvoker.java:296)
    at org.testng.internal.invokers.TestMethodWorker.invokeAfterClassMethods(TestMethodWorker.java:222)
    at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:131)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
    at org.testng.TestRunner.privateRun(TestRunner.java:806)
    at org.testng.TestRunner.run(TestRunner.java:601)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:433)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:427)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:387)
    at org.testng.SuiteRunner.run(SuiteRunner.java:330)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
    at org.testng.TestNG.runSuites(TestNG.java:1099)
    at org.testng.TestNG.run(TestNG.java:1067)
    at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:324)
    at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:74)
    at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:123)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
    at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
    at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)
Caused by: java.lang.AssertionError: expected [false] but found [true]
    at org.testng.Assert.fail(Assert.java:110)
    at org.testng.Assert.failNotEquals(Assert.java:1413)
    at org.testng.Assert.assertFalse(Assert.java:78)
    at org.testng.Assert.assertFalse(Assert.java:88)
    at org.linuxforhealth.fhir.server.test.SearchRevIncludeTest$NutritionOrderCallableDelete.call(SearchRevIncludeTest.java:735)
    at org.linuxforhealth.fhir.server.test.SearchRevIncludeTest$NutritionOrderCallableDelete.call(SearchRevIncludeTest.java:718)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
...

Error:  Failures: 
Error:    SearchRevIncludeTest.testCreateNutritionOrders:495 » Execution java.lang.AssertionError: expected [SUCCESSFUL] but found [SERVER_ERROR]
Error:    SearchRevIncludeTest.testDeleteNutritionOrders:701 » Execution java.lang.AssertionError: expected [false] but found [true]
[INFO] 
Error:  Tests run: 1337, Failures: 2, Errors: 0, Skipped: 113

Environment Which version of LinuxForHealth FHIR Server?

To Reproduce ?

Expected behavior tests should consistently pass or fail

Additional context Add any other context about the problem here.

PrasannaHegde1 commented 1 year ago

There were a couple of NullPointerExceptions in the audit build logs(https://github.com/LinuxForHealth/FHIR/actions/runs/3819990229/jobs/6497976982) which has been fixed as part of pull request: https://github.com/LinuxForHealth/FHIR/pull/4130 .