BurntSushi / quickcheck

Automated property based testing for Rust (with shrinking).
The Unlicense
2.4k stars 149 forks source link

add back a way to put a bound on numbers generated #267

Open BurntSushi opened 3 years ago

BurntSushi commented 3 years ago

It turns out that the tests inside byteorder were designed around the ability to control the bounds of integers generated and there is no easy way to adapt them to quickcheck 1.0 without this ability. I think we can just add a bound mutator on Gen that is by default not set.

dmit commented 3 years ago

Would this bring back gen_range for the new Gen struct, or offer an equivalently powerful solution?

Previously the Gen trait inherited gen_range from the underlying rand::Rng, but now that it's a struct its gen and gen_range methods are private. Would it be possible to make them public? The usefulness of gen_range in particular is evident given how many of the provided Arbitrary impls for std types utilize it.

BurntSushi commented 3 years ago

The intent is to add back methods yes. But I'm not making rand a public dependency.

However, this issue is about something different. This is a configuration knob on Gen itself.

If you would like more methods on Gen, please file new issues with the desired method signatures.

dmit commented 3 years ago

As far as I can tell, making existing private methods on Gen public doesn't expose rand as a dependency. They mirror rand's signatures, but don't leak any implementation details.

But will do!

BurntSushi commented 3 years ago

It does expose rand. Their type signatures include things from the rand crate.

jhpratt commented 3 years ago

Looking through upgrading to 1.0, and this would be great to have.

Presumably it would be possible to accept impl RangeBounds as a parameter in the same way rand does? That wouldn't expose any implementation details, but would allow for using a similar API.

dvc94ch commented 3 years ago

quickcheck seems severely limited without gen_range and gen_bool.

BurntSushi commented 3 years ago

You don't need gen_bool. Arbitrary::bool should work just fine.

It would probably be good to add gen_range, but it doesn't seem particularly difficult to use modulus instead.

BurntSushi commented 3 years ago

And once again, this issue is unrelated to adding helper methods in Gen. This is about bounding values generated by Arbitrary impls for number types.

sdroege commented 3 years ago

I also need this to update some of my code to quickcheck 1.0, specifically to generate floating point numbers in a specific range.

Ralith commented 3 years ago

It would probably be good to add gen_range, but it doesn't seem particularly difficult to use modulus instead.

Floating point values don't have such a convenient hack, unfortunately.

ghost commented 3 years ago

It would probably be good to add gen_range, but it doesn't seem particularly difficult to use modulus instead.

Modulus breaks shrinking. QuickCheck shrinks the generated number, but your number after % can go from MIN_VALUE to MAX_VALUE. In such case, shrinking the number creates a bigger number. This is not at all helpful or useful.

As an example, the values generated in this manner look like this in failures:

assertion failed: `(left == right)`
  left: `(8, 4, 32, 6)`,
 right: `(8, 4, 32, 8)`

The actual failing case, after shrinking, would be either (2, 0, 0, 2) or (1, 59, 59, 29). Other QuickCheck implementations that allow controlling bounds find these cases immediately.

Modulus is completely wrong for this purpose.

BurntSushi commented 3 years ago

@adam-becker I don't know what you're talking about. I was of course referring to the implementation of Arbitrary::arbitrary, not shrinking. This issue has nothing to do with shrinking, which doesn't even get access to a Gen: https://github.com/BurntSushi/quickcheck/blob/defde6fb0ce20b0c8c4e672aa9ae821f7d1f5b38/src/arbitrary.rs#L732-L768

justinlovinger commented 1 year ago

What is the status of this issue? The number of issues from other projects referencing this one is a clear sign people want this resolved.

P.S. A quick check (no pun intended) shows at least one project implemented this functionality on top of QuickCheck, https://github.com/libp2p/rust-libp2p/pull/2857, https://github.com/libp2p/rust-libp2p/blob/master/misc/quickcheck-ext/src/lib.rs. However, I am not sure their implementation is sound when shrinking, and it only works for unsigned numbers.

Colossus commented 1 year ago

Please add back gen_range or equivalent