gchq / Gaffer

A large-scale entity and relation database supporting aggregation of properties
Apache License 2.0
1.77k stars 350 forks source link

Flaky tests in core/graph #3153

Closed omkrpt closed 5 months ago

omkrpt commented 8 months ago

Using the latest commit ac602892435bdd0180f783c80f774149f50eb7f0, the following tests are identified as flaky:

Test 1 : uk.gov.gchq.gaffer.graph.GraphTest#shouldConstructGraphAndCreateViewWithGroups Test 2 : uk.gov.gchq.gaffer.graph.GraphSerialisableTest#shouldConsumeGraph

To Reproduce

I used a tool called nondex.

Test 1:

mvn install -pl core/graph -am -DskipTests
mvn -pl core/graph test -Dtest=uk.gov.gchq.gaffer.graph.GraphTest#shouldConstructGraphAndCreateViewWithGroups
mvn -pl core/graph edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=uk.gov.gchq.gaffer.graph.GraphTest#shouldConstructGraphAndCreateViewWithGroups

Failure:

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.551 s <<< FAILURE! -- in uk.gov.gchq.gaffer.graph.GraphTest
[ERROR] uk.gov.gchq.gaffer.graph.GraphTest.shouldConstructGraphAndCreateViewWithGroups(Store) -- Time elapsed: 0.537 s <<< FAILURE!
org.opentest4j.AssertionFailedError: array contents differ at index [0], expected: <entity2> but was: <entity3>
    at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
    at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
    at org.junit.jupiter.api.AssertArrayEquals.failArraysNotEqual(AssertArrayEquals.java:440)
    at org.junit.jupiter.api.AssertArrayEquals.assertArrayElementsEqual(AssertArrayEquals.java:389)
    at org.junit.jupiter.api.AssertArrayEquals.assertArrayEquals(AssertArrayEquals.java:346)
    at org.junit.jupiter.api.AssertArrayEquals.assertArrayEquals(AssertArrayEquals.java:159)
    at org.junit.jupiter.api.AssertArrayEquals.assertArrayEquals(AssertArrayEquals.java:155)
    at org.junit.jupiter.api.Assertions.assertArrayEquals(Assertions.java:1453)
    at uk.gov.gchq.gaffer.graph.GraphTest.shouldConstructGraphAndCreateViewWithGroups(GraphTest.java:925)
...

Test 2:

mvn install -pl core/graph -am -DskipTests
mvn -pl core/graph test -Dtest=uk.gov.gchq.gaffer.graph.GraphSerialisableTest#shouldConsumeGraph 
mvn -pl core/graph edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=uk.gov.gchq.gaffer.graph.GraphSerialisableTest#shouldConsumeGraph

Failure:

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.586 s <<< FAILURE! -- in uk.gov.gchq.gaffer.graph.GraphSerialisableTest
[ERROR] uk.gov.gchq.gaffer.graph.GraphSerialisableTest.shouldConsumeGraph -- Time elapsed: 0.577 s <<< FAILURE!
org.opentest4j.AssertionFailedError: 

expected: 
  GraphSerialisable[config=GraphConfig[graphId=testGraphId,view=View[{
    "entities" : {
      "e1" : { }
    }
  }],library=uk.gov.gchq.gaffer.store.library.NoGraphLibrary@661d88a,hooks=[uk.gov.gchq.gaffer.graph.hook.NamedViewResolver@4b0b64cc, uk.gov.gchq.gaffer.graph.hook.FunctionAuthoriser@59ce792e]],schema=Schema[{
    "entities" : {
      "e1" : {
        "vertex" : "string"
      }
    },
    "types" : {
      "string" : {
        "class" : "java.lang.String"
      }
    }
  }],properties={gaffer.store.class=uk.gov.gchq.gaffer.integration.store.TestStore, gaffer.store.properties.class=uk.gov.gchq.gaffer.store.StoreProperties}]
 but was: 
  GraphSerialisable[config=GraphConfig[graphId=testGraphId,view=View[{
    "entities" : {
      "e1" : { }
    }
  }],library=uk.gov.gchq.gaffer.store.library.NoGraphLibrary@56399b9e,hooks=[uk.gov.gchq.gaffer.graph.hook.NamedViewResolver@34b9eb03, uk.gov.gchq.gaffer.graph.hook.FunctionAuthoriser@43fda8d9]],schema=Schema[{
    "entities" : {
      "e1" : {
        "vertex" : "string"
      }
    },
    "types" : {
      "string" : {
        "class" : "java.lang.String"
      }
    }
  }],properties={gaffer.store.properties.class=uk.gov.gchq.gaffer.store.StoreProperties, gaffer.store.class=uk.gov.gchq.gaffer.integration.store.TestStore}]
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at uk.gov.gchq.gaffer.graph.GraphSerialisableTest.shouldConsumeGraph(GraphSerialisableTest.java:98)
...

The problem In both of the tests, the objects to be compared contain elements that are arranged in a non-deterministic order.