search

sageserpent-open / americium

Generation of test case data for Scala and Java, in the spirit of QuickCheck. When your test fails, it gives you a minimised failing test case and a way of reproducing the failure immediately.
MIT License
15 stars 1 forks source link

Match error when trying to reproduce a failing test case built with `TrialsApi.uniqueIds`. #71

Closed sageserpent-open closed 8 months ago

sageserpent-open commented 8 months ago

The test support code in the Kinetic Merge repository: MergeTest.simpleMergeTestCases uses trialsApi.uniqueIds.

Trying to reproduce the maximally shrunk failing test case from MergeTest.fullMerge at that Git commit (SHA 65318cdeaaa26ab9467bea7dc61865a14f665641) results in a MatchError in Americium.

scala.MatchError: UniqueId (of class com.sageserpent.americium.generation.UniqueId$)

    at com.sageserpent.americium.TrialsImplementation$$anon$1.apply(TrialsImplementation.scala:103)
    at com.sageserpent.americium.TrialsImplementation$$anon$1.apply(TrialsImplementation.scala:64)

Seen in Americium 1.17.0.

sageserpent-open commented 8 months ago

A cursory look at the two interpreters shows that the one in SupplyToSyntaxSkeletalImplementation (used to generate test cases for a nomal run) has a pattern match for UniqueId, whereas the one in TrialsImplementation (used for test case reproduction) does not - clearly an oversight and lack of testing.

The job is to add an example to the existing Scalatest test for test case reproduction, confirm that it fails and then add the missing pattern match to the second interpreter.

sageserpent-open commented 8 months ago

Done, went out in release 1.18.2, Git commit SHA 53d0ee18eba90b471d351980cc357a143b9ec411.

sageserpent-open commented 8 months ago

Drat - forgot to increment the unique id when it is read, this results in an incorrect reproduction in the aforementioned test in the Kinetic Merge repository. Need to bolster the test and do another fix and release cycle, grrr...

sageserpent-open commented 8 months ago

Another minor release: 1.18.3, Git commit SHA fd6cf1f598a90a67c1ec21fe1cd8f80a0e945860.

sageserpent-open commented 8 months ago

Confirmed that the test in the Kinetic Merge repository now correctly reproduces the maximally shrunk test case.