proptest-rs / proptest

Hypothesis-like property testing for Rust
Apache License 2.0
1.63k stars 152 forks source link

How do I avoid extermely lengthy console outputs when working with large byte vectors? #395

Open vi opened 8 months ago

vi commented 8 months ago

When using inputs like fn frame_roudtrip(s in vec(any::<u8>(), 50..80000)) { ... }, I get very lengthy output for "minimal failing input" section. The output outputs each byte on its own line and in decimal instead of hex.

I can override output one way or another for usual assertions (using e.g. hex_fmt), but how do I inject custom formatter for input without going all the way of defining my own strategies for my own byte vector wrapper just for the debug output?

Or is there maybe some external crate with proptest helpers that e.g. define a type for byte blobs with obvious things already handled?


My current workaround:

fn byte_blob() -> impl Strategy<Value = ByteBlob> {
    vec(any::<u8>(), 50..80000).prop_map(|x|ByteBlob(x))
}

struct ByteBlob(Vec<u8>);

impl std::fmt::Debug for ByteBlob {
    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
        hex_fmt::HexFmt(&self.0).fmt(f)
    }
}
matthew-russo commented 7 months ago

Thanks for reaching out and sorry for the delay.

So as far as how things are today, your current workaround is what I would have suggested.

Currently, its just using pretty-print debug formatter and since its a vec, it ends up looking as you describe: https://github.com/proptest-rs/proptest/blob/dfaa77a5888797b447758eefa5b2cfbcc17efee7/proptest/src/test_runner/errors.rs#L119

Or is there maybe some external crate with proptest helpers that e.g. define a type for byte blobs with obvious things already handled?

Currently we don't another crate with extra utilities. It might be a good idea to keep on the table for the future thought.

What would your ideal interface here look like? I can think of a few different ways of either statically defining different output methods or dynamically configuring a formatter but curious what you had in mind and we can see if it makes sense to implement