sageserpent-open / americium

Generation of test case data for Scala and Java, in the spirit of QuickCheck. When your test fails, it gives you a minimised failing test case and a way of reproducing the failure immediately.
MIT License
15 stars 1 forks source link

Add cases limit strategy to @TrialsTest #47

Closed sageserpent-open closed 2 years ago

sageserpent-open commented 2 years ago

This carries on from #42. Adding an extra field to the @TrialsTest annotation should do the trick, and as this will have to be a name of a declared field, we don't have to worry unduly about Supplier<CasesLimitStrategy> being an erased type at runtime - it's no worse then any other runtime-enforced constraint.

On this topic, is it worth (so soon?) cutting over the signature of withStrategy so it it takes a factory function that yields a CasesLimitStrategy based on some data passed to it that reflects whether the strategy is being used by the initial failure discovery cycle, or by a subsequent shrinkage cycle, and the number of any such shrinkage cycle and whether it is in panic mode or not. Given that #42 has only just gone out, it's probably worth doing this as a hot on the heels breaking change to the API, rather than adding yet more overloads to an already crowded API.

sageserpent-open commented 2 years ago

Amended the API so that a factory function for a CasesLimitStrategy now takes a CaseSupplyCycle that indicates how may previous shrinkage attempts have taken place, thus allowing dynamic configuration.

This has gone out in release 1.6.1, Git commit: 3bc8904ece36902ccc42b00a262dbef87b83b07d .

sageserpent-open commented 2 years ago

Initial design, smoke test and implementation done - there is now an alternative test annotation, @ConfiguredTrialsTest that works with a named field in the test class that is of type TrialsScaffolding.SupplyToSyntax.

This permits full configuration (which has been missing from @TrialsTest) and allows either the older Trials.withLimits overloads, or the new Trials.withStrategy. @TrialsTest is still available and not deprecated, as it is quite handy for quickly throwing things together - @ConfiguredTrialsTest offers the full configuration experience.

The implementation in TrialsTestExtension has got rather messy in the process - it works, but could do with some tidying up.

sageserpent-open commented 2 years ago

TrialsTestExtension is now ported to Scala and hopefully is less of an eyesore.

However, couple of canned implementations of CasesLimitFactory would be good to offer users - one based on a timer, and another that emulates what Scalacheck does with its discard ratio concept.

That should be it for this ticket.

sageserpent-open commented 2 years ago

Now have CasesLimitFactory.counted (which emulates how Scalacheck manages case emission) and CasesLimitFactory.timed.

sageserpent-open commented 2 years ago

This went out in release 1.7.0, Git commit: 989e318fc7892b4c5fcf8677adca4bf4966f81fb