search

TestingResearchIllinois / idoft

38 stars 248 forks source link

gchq/Gaffer,ac602892435bdd0180f783c80f774149f50eb7f0,uk.gov.gchq.gaffer.graph.GraphTest.shouldConstructGraphAndCreateViewWithGroups #1238

Open omkrpt opened 9 months ago

omkrpt commented 9 months ago

Using the current latest commit ac602892435bdd0180f783c80f774149f50eb7f0, the following tests is identified as flaky:

uk.gov.gchq.gaffer.graph.GraphTest#shouldConstructGraphAndCreateViewWithGroups

To Reproduce

mvn install -pl core/graph -am -DskipTests
mvn -pl core/graph test -Dtest=uk.gov.gchq.gaffer.graph.GraphTest#shouldConstructGraphAndCreateViewWithGroups
mvn -pl core/graph edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=uk.gov.gchq.gaffer.graph.GraphTest#shouldConstructGraphAndCreateViewWithGroups

Failure:

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.551 s <<< FAILURE! -- in uk.gov.gchq.gaffer.graph.GraphTest
[ERROR] uk.gov.gchq.gaffer.graph.GraphTest.shouldConstructGraphAndCreateViewWithGroups(Store) -- Time elapsed: 0.537 s <<< FAILURE!
org.opentest4j.AssertionFailedError: array contents differ at index [0], expected: <entity2> but was: <entity3>
    at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
    at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
    at org.junit.jupiter.api.AssertArrayEquals.failArraysNotEqual(AssertArrayEquals.java:440)
    at org.junit.jupiter.api.AssertArrayEquals.assertArrayElementsEqual(AssertArrayEquals.java:389)
    at org.junit.jupiter.api.AssertArrayEquals.assertArrayEquals(AssertArrayEquals.java:346)
    at org.junit.jupiter.api.AssertArrayEquals.assertArrayEquals(AssertArrayEquals.java:159)
    at org.junit.jupiter.api.AssertArrayEquals.assertArrayEquals(AssertArrayEquals.java:155)
    at org.junit.jupiter.api.Assertions.assertArrayEquals(Assertions.java:1453)
    at uk.gov.gchq.gaffer.graph.GraphTest.shouldConstructGraphAndCreateViewWithGroups(GraphTest.java:925)
...

Nondex seed config in txt format: config.txt Execution logs: graphTestFail.txt

What could be the problem? The objects to be compared in the assertion contain elements that are arranged in a non-deterministic order.

omkrpt commented 9 months ago

The PR for the fix is ready for review.

omkrpt commented 9 months ago

Tested this with nondex run in the CI workflow https://github.com/omkrpt/Gaffer/actions/runs/7942664621/job/21686187745

Logs for the run: ci_build_GraphTestWithNondex.txt ci_build_GraphTestWithoutNondex.txt

Here is the diff for the changes made to add nondex. In txt format: ci_config_diff.txt