Automated UI failures in Mac build for 24.0.6

[turkeylurkey]

PR #1258 Attempt #2

• GradleSingleModMPLSTest > testMPQuickFixInJavaFile() FAILED expected: "true" but was: "false" at io.openliberty.tools.intellij.it.TestUtils.validateCodeInJavaSrc (TestUtils.java:314) at io.openliberty.tools.intellij.it.SingleModMPLSTestCommon.testMPQuickFixInJavaFile (SingleModMPLSTestCommon.java:165)
• GradleSingleModMPLSTest > testQuickFixInMicroProfileConfigProperties() FAILED java.net.SocketTimeoutException: Read timed out at io.openliberty.tools.intellij.it.UIBotTestUtils.goToLineAndColumn (UIBotTestUtils.java:1318) at io.openliberty.tools.intellij.it.UIBotTestUtils.pasteOnActiveWindow (UIBotTestUtils.java:1450) at io.openliberty.tools.intellij.it.SingleModMPLSTestCommon.testQuickFixInMicroProfileConfigProperties (SingleModMPLSTestCommon.java:294)
• GradleSingleModMPLSTest > testMPDiagnosticsInJavaPart() FAILED java.net.SocketTimeoutException: Read timed out at io.openliberty.tools.intellij.it.UIBotTestUtils.clickOnFileTab (UIBotTestUtils.java:750) at io.openliberty.tools.intellij.it.SingleModMPLSTestCommon.testMPDiagnosticsInJavaPart (SingleModMPLSTestCommon.java:109)
• GradleSingleModMPLSTest > testDiagnosticInMicroProfileConfigProperties() FAILED java.net.SocketTimeoutException: Read timed out at io.openliberty.tools.intellij.it.UIBotTestUtils.clickOnFileTab(UIBotTestUtils.java:750) at io.openliberty.tools.intellij.it.SingleModMPLSTestCommon.testDiagnosticInMicroProfileConfigProperties (SingleModMPLSTestCommon.java:242)
• GradleSingleModMPLSTest > testInsertMicroProfileProperty() FAILED java.net.SocketTimeoutException: Read timed out at io.openliberty.tools.intellij.it.UIBotTestUtils.clickOnFileTab (UIBotTestUtils.java:750) at io.openliberty.tools.intellij.it.SingleModMPLSTestCommon.testInsertMicroProfileProperty (SingleModMPLSTestCommon.java:185)
• GradleSingleModMPLSTest > executionError FAILED java.net.SocketTimeoutException: Read timed out at io.openliberty.tools.intellij.it.SingleModMPLSTestCommon.cleanup (SingleModMPLSTestCommon.java:55)
• GradleSingleModMPProjectTest > initializationError FAILED com.intellij.remoterobot.utils.WaitForConditionTimeoutException: Exceeded timeout (PT4M) for condition function (IDE did not start) at io.openliberty.tools.intellij.it.SingleModMPProjectTestCommon.prepareEnv (SingleModMPProjectTestCommon.java:905) at io.openliberty.tools.intellij.it.GradleSingleModMPProjectTest.setup (GradleSingleModMPProjectTest.java:78)
• GradleSingleModNLTRestProjectTest > initializationError FAILED com.intellij.remoterobot.utils.WaitForConditionTimeoutException: Exceeded timeout (PT4M) for condition function (IDE did not start) at io.openliberty.tools.intellij.it.SingleModNLTRestProjectTestCommon.prepareEnv (SingleModNLTRestProjectTestCommon.java:225) at io.openliberty.tools.intellij.it.GradleSingleModNLTRestProjectTest.setup (GradleSingleModNLTRestProjectTest.java:47)
• MavenSingleModMPProjectTest > initializationError FAILED com.intellij.remoterobot.utils.WaitForConditionTimeoutException: Exceeded timeout (PT4M) for condition function (IDE did not start) at io.openliberty.tools.intellij.it.SingleModMPProjectTestCommon.prepareEnv (SingleModMPProjectTestCommon.java:905) at io.openliberty.tools.intellij.it.MavenSingleModMPProjectTest.setup (MavenSingleModMPProjectTest.java:86)
• MavenSingleModNLTRestProjectTest > initializationError FAILED com.intellij.remoterobot.utils.WaitForConditionTimeoutException: Exceeded timeout (PT4M) for condition function (IDE did not start) at io.openliberty.tools.intellij.it.SingleModNLTRestProjectTestCommon.prepareEnv (SingleModNLTRestProjectTestCommon.java:225) at io.openliberty.tools.intellij.it.MavenSingleModNLTRestProjectTest.setup (MavenSingleModNLTRestProjectTest.java:45)

71 tests completed, 10 failed

• [ ] #913
• [x] #916
• [ ] #971
• [ ] #972
• [ ] #973
• [ ] #974
• [ ] #991

[turkeylurkey]

I tried with different versions of MacOS to see if that would affect the pass rate. It does. Potentially we should update our build to use these O/S versions.

We could also ask for assistance from github to see if there are parameters to select faster machines to run the Mac tests.

[turkeylurkey]

The GitHub docs explain that the mac-latest O/S runs on M1 with 7GB RAM. This may not be enough for IntelliJ even with fast swap. The macOS-13 and macOS-12 run on x86 which is slower but they get 14GB RAM.

https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners

[turkeylurkey]

Use a configuration action like for Windows but try to get more RAM.

     - name: Configure pagefile
        if: contains(matrix.os, 'windows')
        uses: al-cheb/configure-pagefile-action@v1.2
        with:
          minimum-size: 8GB
          maximum-size: 10GB
          disk-root: "C:"

[turkeylurkey]

To get more RAM maybe we could sign up for a free trial to experiment.

Features not included in the trial

- For GitHub Actions, increased minutes, job concurrency, and larger runners

https://docs.github.com/en/enterprise-cloud@latest/admin/overview/setting-up-a-trial-of-github-enterprise-cloud

[turkeylurkey]

Modified some of the timeouts to increase them to allow more time for the slow Mac VM to respond. In most cases the robot checks every 2s so the increase should not affect other platforms which have successful tests.

[turkeylurkey]

These tests should be fixed by the latest pull request but we should open more issues for new failures and add them to this epic.