sageserpent-open / americium

Generation of test case data for Scala and Java, in the spirit of QuickCheck. When your test fails, it gives you a minimised failing test case and a way of reproducing the failure immediately.
MIT License
15 stars 1 forks source link

Unpicking a trials of a tuple to supply to a multi-parameter Java test. #56

Closed sageserpent-open closed 1 year ago

sageserpent-open commented 1 year ago

There is a pattern in some tests where some pristine expected output is rearranged and / or polluted with other data and then sent to a system under test that should recover the pristine output.

An obvious example is a sorting test, where it is easy to generate an increasing series of values by construction, then shuffle them around to generate input to a sorting algorithm.

Another one is to surround some datum we expect to find with noise - if we put the expected datum and the noise into a collection, we expect to find the datum in the collection.

This pattern is fairly simple to express in Scala tests, because we can built a trials of a pair of the pristine expected output and the messy data, then feed it to a test that is written with an anonymous pattern match:

<pristine expected output trials>.flatMap(pristine => <messy data input trials>.map(pristine -> _))
    .with...(...)
    .supplyTo{case (expected, input) => <test code>}

The JUnit5 integration also provides a type-unsafe way to doing this too for Java folk.

What is missing is the same thing for Java tests that aren't using the JUnit5 integration - some way of expressing the overall pattern.

This might be a way of going from a Trials<Tuple2<X, Y> etc to a Tuple2Trials<X, Y>, which immediately allows a test using a BiConsumer to pick up the two arguments.

Could use a static helper method to do this?

Perhaps Trials.map and Trials.flatMap can be overloaded so that if the transformed type is a Tuple2<x, Y>, the result is a Tuple2Trials<X, Y> rather than the current Trials<Tuple2<X, Y>>. The result can still be turned back into an ordinary Trials<Tuple2<X, Y>> via .trials.

There was an attempt to have Tuple2Trials<x, Y> extend the Trials<Tuple2<X, Y>> interface, but that ended in tears - maybe worth another shot?

sageserpent-open commented 1 year ago

Given that the motivation for not using the JUnit5 integration was to enjoy the strongly-typed coupling between the trials and the parameterised test that is not afforded by the annotation-based approach, this ticket has been ameliorated by #63.

Now it is possible to write a compact test in Java (as well as in Scala) that can be supplied by a triple:

    @TestFactory
    Iterator<DynamicTest> dynamicTestsExampleUsingATriple() {
        final int expectedNumberOfTestCases = 10;

        final TrialsScaffolding.SupplyToSyntax<Tuple3<Integer, String, Boolean>>
                supplier =
                api()
                        .integers()
                        .flatMap(anInteger -> api()
                                .strings()
                                .flatMap(aString -> api()
                                        .booleans()
                                        .map(aBoolean -> Tuple.tuple(anInteger,
                                                                     aString,
                                                                     aBoolean))))
                        .withLimit(expectedNumberOfTestCases);

        final AtomicInteger trialsCount = new AtomicInteger();

        final Iterator<DynamicTest> parameterisedDynamicTests =
                JUnit5.dynamicTests(supplier, (partOne, partTwo, partThree) -> {
                    System.out.format("Test case #%d is %d, %s, %b\n",
                                      trialsCount.incrementAndGet(),
                                      partOne, partTwo, partThree);
                });

        final DynamicTest finalCheck =
                DynamicTest.dynamicTest("Final Check", () -> {
                    assertThat(trialsCount.get(),
                               equalTo(expectedNumberOfTestCases));
                });

        return Iterators.concat(parameterisedDynamicTests,
                                Collections.singleton(
                                        finalCheck).iterator());
    }

Added this test to DemonstrateJUnit5Integration in commit SHA: 7022c4964938d0e24a668aa44608156cb3c39e7a .

As this is pithy and ties into IntelliJ etc while supporting shrinkage, I'll close this ticket.