spockframework / spock

The Enterprise-ready testing and specification framework.
https://spockframework.org
Apache License 2.0
3.52k stars 0 forks source link

Add support for property based testing #533

Open ctapobep opened 8 years ago

ctapobep commented 8 years ago

Currently it's possible only with additional libs that can generate sets of values.

leonard84 commented 8 years ago

Please write a more descriptive user story with motivation, examples and requirements.

ddimtirov commented 7 years ago

To implement property-based testing in quickcheck style, it is important that the data provider is aware of the test outcomes for the previously generated params. Another feature needed is more control over the reported successes and failures (read below)

Using these, it can by testing normal cases, edge cases and extreme numbers, etc (the "expand" phase);

Then once an error is found, do a binary search to identify the surrounding working use cases (shrinking).

And finally as a test outcome, it should be able to report groups of parameter sets, defining the boundaries of a failure (i.e. these cases fail, but these succeed, and they appear to be related), while not reporting the thousands of runs that didn't yield anything useful.