Closed sageserpent-open closed 1 year ago
Some spike code from a client project:
@TestFactory
def foo: util.Iterator[DynamicNode] =
bar(
sourcesTrials and sourcesTrials and sourcesTrials and minimumSizeFractionTrials,
(caze: (FakeSources, FakeSources, FakeSources, Double)) =>
if caze._4 > 0.7 then Trials.reject()
else if caze._4 > 0.5 then throw new RuntimeException("Drat!")
else println(s"Case: $caze")
)
private def bar[Caze](
trials: TrialsScaffolding[Caze],
block: Caze => Unit
): util.Iterator[DynamicNode] =
trials.trials.javaTrials
.withLimit(100)
.testIntegrationContexts()
.asScala
.map(context =>
dynamicTest(
s"Foo - ${customPrettyPrinter(context.caze)}",
() =>
val eligible =
try
context
.inlinedCaseFiltration()
.executeInFiltrationContext(
() => block(context.caze()),
Array(classOf[TestAbortedException])
)
catch
case throwable: Throwable =>
context.caseFailureReporting().report(throwable)
throw throwable
end try
end eligible
if !eligible
then throw new TestAbortedException
end if
)
)
.asJava
Now this is messy because of the need to drop back to the Java API to get at the TestIntegrationContext
instances, but some manual testing shows it does the job.
The plan is to introduce .testIntegrationContexts
into the Scala API form of TrialsScaffolding
so that some extensions can be written, something like:
@TestFactory // JUnit5 dynamic test annotation.
def parameterisedTest: util.Iterator[DynamicTest] =
<trials instance>.withLimit(10).dynamicTests(<parameterised test lambda>)
A nice twist on this if the implementation goes smoothly would be to generalise SupplyToSyntax
by extracting a parent interface, TrialsFactoring.TestIntegrationContexts
- thus allowing both Scala and Java code to enjoy the tighter coupling between the actual trials instance(s) and the test's arguments.
The only reason for the current approach is because JUnit5's @TestTemplate
is geared up to generate test cases outside of the actual test running, and has to use annotations to do this, so one has to live with what Java annotations can support - stringly-typed code.
This went out in release 1.15.1, Git commit SHA: 907ad0d4d719fa0a8c6c84c3824e5c39d68aa325 .
Should add some overloads in the Java and Scala APIs to allow multi-argument parameterised test to be used against a supplier built from several ganged trials.
Also need to update the GitHub Wiki to advertise this.
Support for multi-argument tests went out in release 1.15.2, Git SHA: 9d53fa81375c976f22ed4230ba532ed1dd133bc6 .
So far, if one wants to write a parameterised test in Scala using Americium, the way to do it is to choose a testing framework - so Scalatest, MUnit, uTest and then embed the entire supply within a single test body. This works, but:
Now, it is possible to use the existing JUnit5 integration in Scala code - and this can also pull in Scalatest assertions via scalatestplus-junit5 , but this needs the
Trials
instances to be defined directly in the test class via@TestInstance(Lifecycle.PER_CLASS)
, as the code inTrialsTestExtension
looks for static fields in the test class, not in the Scala companion object instance. Furthermore, theTrials
instances have to be converted to the Java API prior to being used directly via@TrialsTest
, or being ganged together to make a configuredSupplyToSyntax
for use by@ConfiguredTrialsTest
.It all feels a bit hokey - and I personally do not like the loose association between named trials instances and Scala test function arguments; that is forced on folk writing Java as a necessary evil because of the way annotations are defined in Java, but it would be nice to be have proper type-checking on the Scala side.
A spike in Scala yields this:
This yields something similar to the Java JUnit5 integration experience in terms of seeing clickable trials in IntelliJ, but with some finessing could be packaged up into a principled Scala utility.
The use of JUnit5's dynamic tests sacrifices the full test lifecycle support that comes with test templates - so there is no support for
@BeforeEach
/@BeforeAfter
at the level of each trial's execution, but Scala folk used to the Scalatest + Scalacheck integration already have to put up with this. They would use the functional approach to resource management and bracket the test code in either a simple try ... finally block, or use RAII either via the standardUsing
or perhaps Cats'Resource
, so we can settle for just having JUnit5 manage setup / teardown at the suite level or around the entire test factory run.