search

sageserpent-open / americium

Generation of test case data for Scala and Java, in the spirit of QuickCheck. When your test fails, it gives you a minimised failing test case and a way of reproducing the failure immediately.
MIT License
15 stars 1 forks source link

Cause a test failure if no test cases are generated. #65

Closed sageserpent-open closed 9 months ago

sageserpent-open commented 1 year ago

Sometimes no test cases are generated. This an due to:

  1. Filtration via Trials.filter.
  2. Guarding via Trials.whenever.
  3. Post-hoc rejection via Trials.reject.
  4. Too low a complexity limit.

These causes may either completely invalidate all test cases, or perhaps the limit has been set too low in Trials.withLimit leading to premature exhaustion after a run of invalid test cases.

In such a case the test will pass - this isn't good in a continuous integration build setup, as the test is no longer validating anything.

The aim is to have test case supply - either standalone or via the JUnit integrations - throw a special exception when no test cases were produced. There should be a way of switching this off and getting the old behaviour if desired, probably via a fluent method analagous to SupplyToSyntax.withComplexityLimit etc.

sageserpent-open commented 9 months ago

Feature completed as of Git SHA: 992f0a3cb62cfcaf33d69ff1d6dca9fc12f4edbe, scheduled to go out in release 1.18.0.

sageserpent-open commented 9 months ago

This went out in release 1.18.0, Git SHA: 52fe5fd7f56381361c6550d37148d7d5ff97a474.