proptest-rs / proptest

Hypothesis-like property testing for Rust
Apache License 2.0
1.63k stars 152 forks source link

Failed test percentage #436

Open pashadia opened 3 months ago

pashadia commented 3 months ago

Hi all, congrats on a great project!

I'm working on a project which handles a lot of random data, mostly generated from various rand_distr distributions. I would like to implement something like property-based testing, but the particularities of my project allow very few assertions that should be true in all cases. There are a lot more things that "should" hold true in >x% of the cases, but not in 100%.

For instance:

#[cfg(test)]
mod tests {
    use proptest::prelude::*;
    use rand::rngs::mock::StepRng;

    proptest! {
        #[test]
        fn test_random_thing(initial:u64, step:u64) {
            let rng = &mut StepRng::new(initial, step);
            let thing = generate_thing(rng);

            prop_assert!(......);
        }
    }
}

Is there any way to specify that the test assertion should hold true for, e.g. at least 80% of the cases? Would this even be part of what proptest is trying to do?

I would gladly try my hand at an implementation, if this would be accepted as a feature, but doesn't exist yet.

tzemanovic commented 3 months ago

See https://github.com/proptest-rs/proptest/issues/34