sageserpent-open / americium

Generation of test case data for Scala and Java, in the spirit of QuickCheck. When your test fails, it gives you a minimised failing test case and a way of reproducing the failure immediately.
MIT License
15 stars 1 forks source link

Scalacheck Envy #52

Closed sageserpent-open closed 1 year ago

sageserpent-open commented 1 year ago

Look at #9 and doing some spike work on this reveals a few gaps in Americium:

- Picking items from a sequence to make a smaller collection (covered in #21 ?)

Left out of scope for now are distributions - because:

  1. Who really needs them?
  2. Should they be applied in the CaseFactory API, or added as yet more methods / overloads in TrialsApi - presumably to be followed by a map to allow scaling / offsetting.
sageserpent-open commented 1 year ago

Scalacheck also has a probabilistic supplier of Boolean values that favours true with a given probability. It is feasible to replicate this in TrialsApi, but if this is worth doing, then would it be worth extending TrialsApi.chooseWithWeights to work with weights that are long, double, big integer and big decimal. Where do we stop with this?

sageserpent-open commented 1 year ago

Dropping the requirement for BigInt(eger) and BigDecimal as both types have essentially unbounded ranges - so this makes the underlying long input image range used by CaseFactory inadequate to cover the domain.

If one needs smaller domains that would fit into what is covered by long or double values, then it is easier (and more obvious in terms of intent) to just map an existing factory method to the desired type - eg. trialsApi.doubles.map(BigDecimal.apply) in the case of the Scala API.

sageserpent-open commented 1 year ago

Dropping the requirement for Duration for now - it's fairly easy to map from a preferred choice of time unit - milliseconds, seconds, minutes, hours, days etc to either the Java or Scala incarnations. The Scala infinite durations can be mixed in via alternation.

sageserpent-open commented 1 year ago

Dropping the requirement for stream cases - the existing collection helpers can create collection cases of arbitrarily large size, and these can be mapped to the desired stream type.

sageserpent-open commented 1 year ago

Sampling of single test cases can be performed by obtaining an iterator via the Java API (which is accessible from the Scala API via .javaTrials) and hitting that for successive test cases (which may still be Scala-specific types).

This makes the implied state change required to get varying test cases on each pull explicit, as well as the point at which the state is reset for a new sequence of test cases - namely the call to Trials.withLimit/withStrategy.asIterator.

It would be a courtesy to Scala folks to mirror SupplyToSyntax.asIterator/testIntegrationContexts in the Scala API, though...

sageserpent-open commented 1 year ago

Doing some groundwork to expose SupplyToSyntax.asIterator/testIntegrationContexts in the Scala API results in a problem in the implementation of SupplyToSyntaxSkeletalImplementation - this has to provide overrides that simultaneously override both the Scala and Java API definitions.

Now, that has been done for the likes of supplyTo, but would require some irritating footwork wherein the single override for each method pair would have to yield an iterator extending both the Scala and Java forms of Iterator. This is probably possible by writing a custom variation on IteratorWrapper/JIteratorWrapper from the conversions framework, but is awkward.

Rather than devote effort to this, let's see if the need arises for a direct Scala API where we say .withLimit/withStrategy.asIterator as opposed to using .javaTrials.withLimit/withStrategy.asIterator.asScala.

sageserpent-open commented 1 year ago

Completed support for impossible cases in SHA: 387828466eb01a561b1dd7104afa2e40bd7b1342 .

sageserpent-open commented 1 year ago

Reinstating the requirement for BigInt(eger) and BigDecimal as the implementation can now support bounded trials.

sageserpent-open commented 1 year ago

Implemented in Git commit SHA: 279c2c1cf2d679d500c935ce05db4305c7e99d40 .

sageserpent-open commented 1 year ago

This went out in release 1.10.4, Git commit SHA: 5af012697fbd1065889803a7c2c80dc2d8c259ca .