Resolve Linux build failures introduced by PR 553

[TrevCraw]

https://github.com/OpenLiberty/liberty-tools-intellij/pull/553 appears to have introduced failures in the Linux test runs. Linux build results for that PR can be found here: https://github.com/OpenLiberty/liberty-tools-intellij/actions/runs/6690117829/job/18211772385

A subsequent PR, https://github.com/OpenLiberty/liberty-tools-intellij/pull/562, consistently hit the same failures for its Linux build.

The code seems correct for https://github.com/OpenLiberty/liberty-tools-intellij/pull/553, since both the Mac and Windows tests were passing. It would appear that the code change is triggering some sort of issue with the tests that is unique to the Linux builds.

[mrglavas]

On investigation of the test failures we discovered (by reviewing the video from a failing test) that configuration from previous tests started bleeding into the start dev mode in container tests as a result of PR 553. Specifically the -DhotTests option is being passed in and I have learned that has been problematic before with our automated testing. While we continue investigating how to properly address this issue with the tests, I would suggest that we disable the affected tests so that we get clean builds again.

[anusreelakshmi934]

Ran the GHA builds for linux only. The tests which are consistently failing ( 6/10 ) is MavenSingleModMPProjectTest > testCustomStartParametersClearedOnConfigRemoval() . Due to the error -org.opentest4j.AssertionFailedError: Debug Port is not set to 9876 ==> expected: <true> but was: <false>. When inspected the video it only has the portion upto where the configuration is added. And then it fails. I tried adding delay thinking that it may be failing beacuse it checks for port before starting the configuration. But that was not successful

Attaching the build link - https://github.com/anusreelakshmi934/liberty-tools-intellij/actions/runs/10465042438

[anusreelakshmi934]

Also created a new branch from lsp4ij-market-0.3.0-integration and ran tests in that. Out of 10 linux builds that ran, two of them failed. One of it had timed out errors. The other build had a failure in MavenSingleModMPProjectTest > testCustomStartParametersClearedOnConfigRemoval() FAILED org.opentest4j.AssertionFailedError: Debug Port is not set to 7777 ==> expected: but was: When looked in the video found that 7777 port was not available so instead it used another port.

Both of them were success in the rerun.

https://github.com/anusreelakshmi934/liberty-tools-intellij/actions/runs/10465306663/attempts/1

[anusreelakshmi934]

In some of the tests the devc command is already present before the start of the test.

One way to ensure that the tests start with a fresh server, the server should be stopped before each test. However, the stop command only works after the server has fully started. When the tests which failed begin, the server hasn't fully started, causing issues. Although the q command can be used to stop dev mode manually, it doesn't work properly during testing. Instead of stopping the server, the test just keeps sending the q command without success.

[turkeylurkey]

I reviewed the video for run #6 (which failed) to see what happened during the testCustomStartParametersClearedOnConfigRemoval() test. The first server start uses debug port 9876 which runs ok. Then the config is removed ok and the second server start just uses the default debug port 7777 which would be ok except there is still a server running from a previous error. So this test is correct and the problem is with a previous test.

[turkeylurkey]

In another video from test run 6 we see the stop command issued correctly but the server does not stop even after waiting 10 minutes. Working backwards we see the previous server start did not detect the server start string within the 90s limit in liberty tools. And the command before that was a devc command. Need to examine the logs to see what test was before testRunTestsActionUsingPopUpMenu_recording_2024_20_08_05_32_25.

[anusreelakshmi934]

In another video from test run 6 we see the stop command issued correctly but the server does not stop even after waiting 10 minutes. Working backwards we see the previous server start did not detect the server start string within the 90s limit in liberty tools. And the command before that was a devc command. Need to examine the logs to see what test was before testRunTestsActionUsingPopUpMenu_recording_2024_20_08_05_32_25.

@turkeylurkey . I saw this in another build. So I ran a GH build by capturing all the test videos to find how the previous test was causing this error. https://github.com/anusreelakshmi934/liberty-tools-intellij/actions/runs/10350980498/job/28648500130 In this testStartWithConfigInDebugModeUsingToolbar test fails due to this devc already present in the terminal issue. So When I checked the previous test which uses container it was testStartInContainerActionUsingSearch. But when looking into the video it seems to be incomplete. But the test ran success. Tried to reproduce this locally but did not encounter this error.

[anusreelakshmi934]

Linux Builds results - https://ibm.ent.box.com/notes/1628253114811

[anusreelakshmi934]

https://github.com/anusreelakshmi934/liberty-tools-intellij/actions/runs/10350980498/job/28648500130 - In this build we can see that MavenSingleModMPProjectTest > testStartWithConfigInDebugModeUsingToolbar() FAILED due to devc already present in the terminal. When we look into the previous container test which ran it was MavenSingleModMPProjectTest > testStartInContainerActionUsingSearch() STANDARD_OUT which was success. But the video seems to be incomplete. We can find the videos here. - https://github.com/anusreelakshmi934/liberty-tools-intellij/actions/runs/10350980498/artifacts/1801858116

[turkeylurkey]

Maybe that video is complete. After it issues the start in container command it connects with a server and verifies there is a server running. Maybe there is still a server running from a previous test. Can you go back to the test before the second testStartInContainerActionUsingSearch and see if it finished correctly? That is go back to the previous last test where a server was started.

Also, could you add the "all videos" config to the development.md in any pull request for this issue or just make a new issue for a pull request specifically for adding that config? Thanks.

[anusreelakshmi934]

The video of testStartInContainerActionUsingSearch in maven project is not complete. I cross checked with the gradle project video. Also The test before testStartInContainerActionUsingSearch() was testStartWithParamsActionUsingPlayToolbarButton(). This finished correctly. Therby confirming the problem lies within the Start in container test.

[anusreelakshmi934]

The tests are failing because the devc command from a previous test is still running. In the build logs, we can see that the testStartWithConfigInDebugModeUsingToolbar test fails because devc is already running in the terminal, carried over from the previous test, testStartInContainerActionUsingSearch.

Even though the logs indicate that testStartInContainerActionUsingSearch was successful, the video recording suggests that the test did not fully complete. Additionally, when checking the terminal logs, there are no logs from testStartInContainerActionUsingSearch. Instead, the last logs in the terminal are from the testStartWithParamsActionUsingPlayToolbarButton test.

By examining the videos, we can see that testStartWithParamsActionUsingPlayToolbarButton finished at 2024-08-12T12:22:10Z. When reviewing the video for testStartInContainerActionUsingSearch, the same line is still visible in the terminal, indicating that testStartInContainerActionUsingSearch did not actually run in that terminal.

[anusreelakshmi934]

Next steps to be tried in this issue are :

• Add steps in test to close the project after each of the test and then reopen in the next test.
• Add step to close the terminal window after each test.