warn users that panic hooks may interfere with failure messages

[llogiq]

When using quickcheck in Tests (cargo test), I only learn that there was a failure, but not the failing inputs.

Perhaps adding log output and giving instructions in the README would be a quick solution?

[BurntSushi]

Could you show the output that you're looking at? It should show the inputs. Note this code calls the method failed_msg, which includes the arguments generated.

Consider this code:

extern crate quickcheck;

pub fn reverse<T: Clone>(xs: &[T]) -> Vec<T> {
    let mut rev = vec!();
    for x in xs.iter().skip(1) {
        rev.insert(0, x.clone())
    }
    rev
}

#[cfg(test)]
mod tests {
    use super::reverse;
    use quickcheck::quickcheck;

    #[test]
    fn reverse_prop() {
        fn p(xs: Vec<isize>) -> bool {
            xs == reverse(&reverse(&xs))
        }
        quickcheck(p as fn(Vec<isize>) -> bool);
    }
}

Running cargo test produces:

$ cargo test
   Compiling qc-138 v0.1.0 (file:///tmp/qc-138)
     Running target/debug/qc_138-af8faaf8d4e6cf17

running 1 test
test tests::reverse_prop ... FAILED

failures:

---- tests::reverse_prop stdout ----
        thread 'tests::reverse_prop' panicked at '[quickcheck] TEST FAILED. Arguments: ([0])', /home/andrew/.multirust/toolchains/nightly/cargo/registry/src/github.com-88ac128001ac3a9a/quickcheck-0.2.27/src/tester.rs:116
note: Run with `RUST_BACKTRACE=1` for a backtrace.

failures:
    tests::reverse_prop

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured

Note the section: thread 'tests::reverse_prop' panicked at '[quickcheck] TEST FAILED. Arguments: ([0])'.

[llogiq]

My bad, somehow panic::set_hook(_) / take_hook(_) failed to reset the panic handler as advertised, so the panic containing the inputs was swallowed.

Perhaps warning of such a situation in the README would help others avoid this issue.

[mulkieran]

@llogiq @BurntSushi Hi! I'm having the same problem, but I don't really understand the discussion. There is one difference, I'm not using the macro, because I want to limit the number of tests:

        QuickCheck::new().tests(30).quickcheck(denominator_result as fn(u32) -> bool);

But the error looks just the same:

[mulhern@dhcp-25-209 stratisd]$ make test
cargo test
    Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs
     Running target/debug/stratisd-1890f69058a37c6f

running 1 test
test sim_engine::randomization::tests::prop_denominator_result ... FAILED

failures:

---- sim_engine::randomization::tests::prop_denominator_result stdout ----
    thread 'sim_engine::randomization::tests::prop_denominator_result' panicked at '[quickcheck] TEST FAILED. Arguments: (1)', /home/mulhern/.cargo/registry/src/github.com-1ecc6299db9ec823/quickcheck-0.4.0/src/tester.rs:118
note: Run with `RUST_BACKTRACE=1` for a backtrace.

failures:
    sim_engine::randomization::tests::prop_denominator_result

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured

error: test failed
Makefile:13: recipe for target 'test' failed
make: *** [test] Error 101

Can you tell me what I should do?

Edit: Oh, sorry. I see that I can interpret the error message, the bad value is 1.