vaticle / typedb-benchmark

TypeDB Simulation and Benchmarking Library
GNU Affero General Public License v3.0
9 stars 15 forks source link

Simulation determinism is dependent on non-functional code structuring #145

Open james-whiteside opened 1 year ago

james-whiteside commented 1 year ago

Description

The following lines are present in the Simulation class:

val randomSource2 = randomSource.nextSource()
if (context.recreateDatabase) init(randomSource2)

Different data is generated when replaced with the following lines:

if (context.recreateDatabase) init(randomSource.nextSource())
else randomSource.nextSource()

The following unit test was written to test the behaviour of the RandomSource class:

@Test
fun testSource() {
    val seed: Long = 0
    val randomSourceA = RandomSource(seed)
    val randomSourceB = RandomSource(seed)
    assert(randomSourceA.nextInt(1000000) == randomSourceB.nextInt(1000000))

    val randomSourceA2 = randomSourceA.nextSource()
    repeat(1000000) {
        randomSourceA2.nextBoolean()
    }

    if (false) {}
    else randomSourceB.nextSource()

    assert(randomSourceA.nextInt(1000000) == randomSourceB.nextInt(1000000))
}

The test passed. It is unknown why a change that appears to be a trivial refactor causes changes to the data generated, and this warrants looking further into at a later date.

Environment

  1. OS: MacOS 12.6.1
  2. TypeDB version: 2.16.1
  3. TypeDB client: client-java