Closed sageserpent-open closed 10 months ago
NOTE: This isn't a bug in TrialsAPi.complexities
- that is doing precisely what it should do, namely peeking at the complexity introduced by the surrounding flatmapping context. The act of peeking shouldn't add more complexity, and indeed it does not.
EDIT: there is a related bug that has been incorporated into this ticket, see below.
While using the workaround shown above in a real test - changes and link to browse sources here, I noticed that using a recipe to reproduce the maximally shrunk failing test case causes Americium to fault with a match error...
scala.MatchError: List() (of class scala.collection.immutable.Nil$)
at com.sageserpent.americium.TrialsImplementation.com$sageserpent$americium$TrialsImplementation$$anon$1$$_$apply$$anonfun$1(TrialsImplementation.scala:75)
at cats.data.IndexedStateT.map$$anonfun$1(IndexedStateT.scala:58)
at cats.data.IndexedStateT.transform$$anonfun$1$$anonfun$1$$anonfun$1(IndexedStateT.scala:122)
at cats.Eval.map$$anonfun$1(Eval.scala:78)
at cats.Eval.cats$Eval$$anon$2$$_$$lessinit$greater$$anonfun$2(Eval.scala:104)
at cats.Eval$.loop$1(Eval.scala:341)
at cats.Eval$.cats$Eval$$$evaluate(Eval.scala:384)
at cats.Eval$FlatMap.value(Eval.scala:305)
at com.sageserpent.americium.TrialsImplementation.com$sageserpent$americium$TrialsImplementation$$reproduce(TrialsImplementation.scala:108)
at com.sageserpent.americium.TrialsImplementation$SupplyToSyntaxImplementation$1.reproduce(TrialsImplementation.scala:274)
at com.sageserpent.americium.generation.SupplyToSyntaxSkeletalImplementation.testIntegrationContextReproducing$1(SupplyToSyntaxSkeletalImplementation.scala:891)
at com.sageserpent.americium.generation.SupplyToSyntaxSkeletalImplementation.$anonfun$15(SupplyToSyntaxSkeletalImplementation.scala:933)
Given that the test uses complexities to drive the weighting of alternative trials, and given this code that hardwires the complexity to zero when reproducing a test case from a recipe, this is clearly going to result in an inconsistency between the flatmapping context of trials and the decision stages deserialised from the recipe.
So while we're on this ticket, let's fix that bug into the bargain.
The bug is has been fixed at Git commit SHA: 12cfdbd65d8c4f36e5711cdbc989ced4ad0f09aa.
The bug-fix and following introduction of TrialsApi.uniqueIds
went out in release 1.17.0, Git SHA: 63d7bc6f9ef528714cd468222f6e8c5496dcb357.
According to the wiki entry here, complexities can be used as a source of unique ids when generating a complex test case with extensible internal structure.
That example works, but consider this:
This yields:
That surprised me!
What happens is that complexity picks up each choice or factory picked in the surrounding context of flatmapping that led to the call of
TrialsApi.complexities
.For
threeComplexities
, there are neither choices nor factories involved, and the complexities are just three zeroes.For
recursiveComplexities
, there are implicit choices involved in the recursion viaapi.alternate
, so while we still see three equal complexities, they do vary from one case to another.For
accumulatedRecursiveComplexities
, we see something that almost satisfies the requirement for unique ids, but still each triple has equal complexities.The workaround is to define something like this:
That yields:
The job here is to package this workaround up in
TrialsApi.uniqueIds
, and go for a cleaner, more direct implementation - probably another case class belonging to theGenerationOperation
hierarchy.