Gathering and displaying statistics

[Xion]

QuickCheck allows to obtain some statistics about test cases that have been generated via classify and collect functions. pyqcy should have them too.

What those functions do can be seen e.g. here. The bottom line is:

• collect takes a value and creates a "histogram" of test cases that share the same value. For example collect (length s) will divide cases based on length of the s string and then display this information (as absolute and percentage quantities).
• classify allows to categorize test cases by putting them into named "buckets", based on some condition. For example we could have classify (length s == 0) "empty string" along with classify (length s > 0 && length s < 10) "short string" and classify (length s >= 10) "long string". Of course, conditions can get arbitrarily complex and involve many variables.

In QuickCheck this functionality is done through function composition. In pyqcy, we will use yield as a sort of "system call" inside properties. Proposed syntax:

from pyqcy import *

@qc
def sort_works(
    l=list_(of=int_, max_length=64)
):
    yield collect(len(l))
    yield classify(sum(l) > 0, "sums to positive")
    yield classify(sum(l) < 0, "sums to negative")
    assert sorted(l)[0] == min(l)

Expected results:

sort_works: passed 100 tests:
1.54% 0
1.54% 1
...
1.54% 64

---
51% sums to positive
49% sums to negative

[Xion]

Correction: The expected result should group collect and classify values into one, i.e.:

sort_works: passed 100 tests.
0.77% 0, sums to positive
0.77% 0, sums to negative
0.77% 1, sums to positive
0.77% 1, sums to negative
...
0.77% 64, sums to positive
0.77% 64, sums to negative

[Xion]

Implemented.