Closed sageserpent-open closed 2 years ago
Owing to the restriction on Java annotations, TestTrials
has to use a similar approach to ParameterizedTest
in how it specifies its trials instances - namely to supply the names of the fields in the class housing the tests that are typed as Trials
.
Now this leads to a situation where one would have to provide a field purely to allow a definition of a conjoined trials that is defined by the and
combinator - and if several tests make use of varying combinations of the same basic trials instances, each combination would need its own field so that the corresponding @TestTrials
would have something to name. This feels awkward - so again in the spirit of ParameterizedTest
, TestTrials
can take multiple names of trials fields and will do its own internal joining, rather than using a Trials
built externally via and
. The test method then takes the corresponding number of arguments.
It is possible that folk may wish to supply either a single such externally conjoined trials instance and have it unpicked to drive a test method with multiple arguments - or perhaps a single tuple argument should be supplied? What happens if both approaches are mixed and matched - do we look for tuple types in a multi-argument test method and supply them directly from the corresponding conjoined trials, and if we have a run of non-tupled arguments, do we supply them from a single conjoined trials and then carry on as usual with the rest of the arguments? This needs some thought ....
What exists right now is workable - where inlined filtration is used, JUnit interprets a rejected test case in the same way as if a call to assumeTrue
etc failed - it marks the individual test as rejected.
Indeed, one can mix both the use of Trials.whenever
and the Assumptions.assume*
methods - both are picked up by the JUnit integration furnished by a @TestTrials
annotation on a test method. The usual test before and after methods are also supported, as is selecting an individual test case in IntelliJ to run it.
Merged into master
in commit 6921fe703f0e629df31ac9bafb7f6a4f27f1167a, released in 1.3.0 .
This follows on from the train of work from issue #18, which had been repurposed as a bug-fix ticket.
Ongoing spike work started off the back of that ticket indicates that it is worth considering a tighter integration with JUnit5, whereby something akin to the
ParameterizedTest
annotation can be provided using theTestTemplate
framework (also used byParameterizedTest
, which looks to be too specialised to be amenable to integration withTrials
, not least because implementing inlined filtration won't play well with its notion of pulling from a stream ofArguments
- it doesn't provide a context for theInlinedCaseFiltration
to be available when the test instances are run.What the spike is investigating right now is whether an
InlinedCaseFiltration
can be provided as a context by embedding an iterator over cases and the inlined case filtration within an extension that implementsTestTemplateInvocationContextProvider
andInvocationInterceptor
...If so, then the end result will be
TrialsTest
annotation that couples to an instance ofTrials
, supplying the test method with arguments that align with the cases yielded by the trials instance - so a simple trials will support a single-argument test, whereas one constructed by a single application of.and
will support a test with two arguments, etc.It is hoped that using
InvocationInterceptor
will allow inlined case filtration.A goal for the future is to build on this ticket's deliverable functionality so that once a test case fails, further test cases will be generated using the existing shrinkage logic - so whereas
ParameterizedTest
always generates the same test cases for deterministic value sources,TrialsTest
only does this when all test cases pass - if there are failures then the shrinkage kicks in and will vary depending on the first failing test case. This is desirable, as when we have a failure we simply want to work on a minimal test case, and as long as the mode of failure remains the same then we will have repeatable failing tests leading to the same minimal test case, which hopefully will be amenable to using IntelliJ to run it directly without recapitulating the test cases leading to it.