Since Wednesday 27th March there has been a large number of our integration tests failing. Every day we get a handful of the integration tests that run the ZosManagerBatchIVT failing.
The integration tests run log looks like this:
15/04/2024 06:12:55.445 INFO main dev.galasa.galasaecosystem.internal.LocalLinuxEcosystemImpl.submitRun - Submitted test run with run name of TESTZM1
15/04/2024 06:13:31.232 DEBUG GalasaSSHClient timeout thread dev.galasa.ipnetwork.spi.SSHClient.run - SSH Client unused after 120000 milliseconds, freeing session
15/04/2024 06:14:51.248 DEBUG GalasaSSHClient timeout thread dev.galasa.ipnetwork.spi.SSHClient.run - SSH Client unused after 120000 milliseconds, freeing session
15/04/2024 06:15:55.600 INFO main dev.galasa.framework.GenericMethodWrapper.invoke - Ending
----------------------- ****************************************************************************************************
----------------------- *** Failed - Test method dev.galasa.inttests.zosBatch.local.mvp.ZosBatchLocalJava11UbuntuMvpRse#testZosBatchIvtTestRSE,type=Test
----------------------- ****************************************************************************************************
dev.galasa.galasaecosystem.GalasaEcosystemManagerException: Unable to determine the status of run name TESTZM1
at dev.galasa.galasaecosystem.internal.LocalEcosystemImpl.waitForRun(LocalEcosystemImpl.java:428)
at dev.galasa.galasaecosystem.internal.LocalEcosystemImpl.waitForRun(LocalEcosystemImpl.java:385)
at dev.galasa.inttests.zosBatch.AbstractZosBatchLocalRSE.testZosBatchIvtTestRSE(AbstractZosBatchLocalRSE.java:34)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at dev.galasa.framework.GenericMethodWrapper.invoke(GenericMethodWrapper.java:88)
at dev.galasa.framework.TestMethodWrapper.invoke(TestMethodWrapper.java:61)
at dev.galasa.framework.TestClassWrapper.runTestMethods(TestClassWrapper.java:213)
at dev.galasa.framework.TestRunner.runTestClassWrapper(TestRunner.java:607)
at dev.galasa.framework.TestRunner.runEnvironment(TestRunner.java:581)
at dev.galasa.framework.TestRunner.createEnvironment(TestRunner.java:546)
at dev.galasa.framework.TestRunner.generateEnvironment(TestRunner.java:503)
at dev.galasa.framework.TestRunner.runTest(TestRunner.java:424)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at dev.galasa.boot.felix.FelixFramework.runTest(FelixFramework.java:234)
at dev.galasa.boot.Launcher.launch(Launcher.java:176)
at dev.galasa.boot.Launcher.main(Launcher.java:126)
Caused by: dev.galasa.galasaecosystem.GalasaEcosystemManagerException: Run name TESTZM1 did not finish in time
at dev.galasa.galasaecosystem.internal.LocalEcosystemImpl.waitForRun(LocalEcosystemImpl.java:426)
... 21 more
The IVT's run log:
The run log just repeats lots of logging showing GET requests to a mainframe then 200 responses, then the run log just ends. No exceptions or errors. I can't post the run log in this story but some runs that have had this failure that can be downloaded with galasactl are:
C786
C809
C815
There are six integration tests that run the ZosManagerBatchIVT and as not all of these six tests fail every day it can't be an error with the IVT or Manager that is guaranteed to happen every time the test is ran.
ZosBatchLocalJava11UbuntuRse
ZosBatchLocalJava11UbuntuIsolatedRse
ZosBatchLocalJava11UbuntuMvpRse
ZosBatchLocalJava11UbuntuZosmf
ZosBatchLocalJava11UbuntuIsolatedZosmf
ZosBatchLocalJava11UbuntuMvpZosmf
Ideas
???
Steps to reproduce
Prepare a portfolio of tests
Remove all except the 6 above
Submit the portfolio
See error
Expected behavior
The tests should all Pass.
Additional context
If applicable, add screenshots or any relevant output logs to help explain your problem.
Describe the bug
Since Wednesday 27th March there has been a large number of our integration tests failing. Every day we get a handful of the integration tests that run the ZosManagerBatchIVT failing.
The integration tests run log looks like this:
The IVT's run log: The run log just repeats lots of logging showing GET requests to a mainframe then 200 responses, then the run log just ends. No exceptions or errors. I can't post the run log in this story but some runs that have had this failure that can be downloaded with
galasactlare:There are six integration tests that run the ZosManagerBatchIVT and as not all of these six tests fail every day it can't be an error with the IVT or Manager that is guaranteed to happen every time the test is ran.
Ideas
???
Steps to reproduce
Expected behavior
The tests should all Pass.
Additional context
If applicable, add screenshots or any relevant output logs to help explain your problem.