BurntSushi / quickcheck

Automated property based testing for Rust (with shrinking).
The Unlicense
2.4k stars 149 forks source link

Shrinking only appears to consider one "layer" of shrinking. #127

Closed cstorey closed 8 years ago

cstorey commented 8 years ago

I'm used to clojure's test.check, which uses a recursive approach to shrinking, and I think it would make a nice addition, and is especially useful when generating a series of commands. For example, input to a state machine that models a replication instance.

For example, if we have a (hypothetical) sequence of operations modeled as booleans like: [true, false, false, true, false, true], and we want to find the smallest sequence that contains more than 2 true instances, then we'd like to be able to reduce it down to a single [true, true, true] instance. Concretely:

    #[test]
    fn testy_test() {
        use quickcheck::Arbitrary;
        fn test_can_read_prepared_values_qc(vals: Vec<bool>) -> bool {
            vals.iter().filter(|&v| *v).count() < 3
        }
        quickcheck(test_can_read_prepared_values_qc as fn(vals: Vec<bool>) -> bool)
    }

At the moment, this results in (for example): thread 'test::testy_test' panicked at '[quickcheck] TEST FAILED. Arguments: ([false, true, true, true, true, false, false, false, true, false, true, false, true, false, false, true, false, false, false, false, true, true, true, false, false, false, true, false, true, true, false, true, true, true, true, true, true, true, true, false, true, false, true, true, false, false, true, false, true, true, true, false, false, true, true, true, true, false, true, false, true, true, true, true, true, true, false, true, true, false, false, false, false, false, false, false, false])', /home/ceri/2016/03/09/quickcheck/src/tester.rs:116

My feeling is that this shouldn't require too much surgery, should be confined to the Testable instances for functions.

cstorey commented 8 years ago

As it turns out, the old code did this, by just recursing through testable::shrink_failure.

cstorey commented 8 years ago

Fixed in #129