BurntSushi / quickcheck

Automated property based testing for Rust (with shrinking).
The Unlicense
2.4k stars 149 forks source link

Port over the shrinking behavior from before f017c01. #129

Closed cstorey closed 8 years ago

cstorey commented 8 years ago

Re: tickets #126 and #127.

This ensures that we try to recursively shrink a failing case, in order to find the smallest instance. Repeating myself, this means that the test case:

#[cfg(test)]
mod test {
    use quickcheck::quickcheck;
    #[test]
    fn testy_test() {
        use quickcheck::Arbitrary;
        fn thetest(vals: Vec<bool>) -> bool {
            vals.iter().filter(|&v| *v).count() < 2
        }
        quickcheck(thetest as fn(vals: Vec<bool>) -> bool)
    }

Will fail with [quickcheck] TEST FAILED. Arguments: ([true, true]), rather than something like [quickcheck] TEST FAILED. Arguments: ([false, false, false, true, true, true, false, true, true, ...]). This was mostly just adapting the logic used previously to the new macro.

I'd welcome any feedback on how you think it might be improved.

Thanks,

cstorey commented 8 years ago

FWIW, the tests seem to be failing thanks to some oddities with how travis deploy the rust compiler. They pass on both nightly and stable locally.

BurntSushi commented 8 years ago

Thanks! Is there any chance we can add a regression test for this? I've found it pretty tricky in general to test this stuff, but one way might be to take your test and run it with quicktest (which returns a Result<usize, TestResult>, unlike the typically more convenient quickcheck). You'd then have to inspect the Vec<String> arguments: http://burntsushi.net/rustdoc/src/quickcheck/tester.rs.html#130-134

If not, no worries and I'll get this merged today.

cstorey commented 8 years ago

I'll more than likely have some time to do that tomorrow, then.

cstorey commented 8 years ago

Okay, I've added the test; but I'll happily admit that it is basically a hack; hypothetically, I could parameterize the TestResult on the argument-tuple type, but that'd be a significant change for what seems like comparatively little value.

Thanks,

BurntSushi commented 8 years ago

@cstorey Awesome, that looks like exactly what I had in mind! Thank you! :-)

BurntSushi commented 8 years ago

This PR is in 0.2.26 on crates.io.