search

gnzlbg / bitwise

Portable high-level bitwise manipulation algorithms
https://gnzlbg.github.io/bitwise
MIT License
48 stars 9 forks source link

Introduce quickcheck testing #15

Closed blt closed 7 years ago

blt commented 7 years ago

This commit introduces the quickcheck library into the project, testing the word/is* traits. Quickcheck is very helpful for finding edge-case bugs in complex algorithms -- of which the is* are not -- so this commit exists as more of a demonstration of technique. Macros are used to generate test code over built-in integer types and slower but "obviously correct" algorithms are tested against the production quality code.

I'd like to use bitwise in a compact datastructure project which I expect to QC and fuzz like crazy. To that end I thought it wise to introduce QC'ing directly into bitwise and, if you're open to it, fuzzing as well via cargo-fuzz in a latter commit. QC testing is less efficient at detecting bugs compared to exhaustive testing where exhaustive testing is applicable but pretty good at searching the haystack in large domains like u64. This can be further improved with branch-directed fuzzing.

I'll add more QC tests over time.

Signed-off-by: Brian L. Troutwine brian@troutwine.us

coveralls commented 7 years ago

Coverage Status

Coverage increased (+100.0%) to 100.0% when pulling 4a9eb2a24af0c7245bd47697df5711c33a84d346 on blt:introduce_qc into 12dc77da62af8c043ab6a5d50729ec329fdff536 on gnzlbg:master.

gnzlbg commented 7 years ago

@blt Thanks for the PR! Looks good!

gnzlbg commented 7 years ago

First, thanks again. In general, I prefer to "brute-test" the algorithms against all possible values, but as you mention, this can't be done for 64 bit, and usually can't also be done for algorithms taking multiple values (not even for 32-bit). The GCD algorithms have these problems, and I've wanted to learn and introduce quickcheck since. Thank you for beating me to it, now I have no excuse to learn it.

I think that using cargo-fuzz is a nice idea, feel free to experiment with it and send a PR. I've opened issue #16 with my thoughts on the tooling that I would like to have in place for a 1.0 release.

blt commented 7 years ago

I agree, when possible exhaustive testing is ideal. I'm hoping to get a little time this weekend to QC some more of the complex algorithms present in the library.

Will do, with regard to cargo-fuzz. Concerning #14, why not rely on https://docs.rs/bitwise/0.1.1/bitwise/ ?

gnzlbg mailto:notifications@github.com February 27, 2017 at 1:32 AM

First, thanks again. In general, I prefer to "brute-test" the algorithms against all possible values, but this can't be done for 64 bit, and usually can't also be done for algorithms taking multiple values (not even for 32-bit). The GCD algorithms have these problems, and I've wanted to learn and introduce quickcheck since. Thank you for beating me to it, now I have no excuse to learn it.

I think that using cargo-fuzz is a nice idea, feel free to experiment with it and send a PR. I've opened issue #16 https://github.com/gnzlbg/bitwise/issues/16 with my thoughts on the tooling that I would like to have in place for a 1.0 release.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/gnzlbg/bitwise/pull/15#issuecomment-282670292, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAN-5YHYzMhSoVCml5uAYLeXzMyNTtCks5rgphDgaJpZM4MMpYb.

gnzlbg commented 7 years ago

Will do, with regard to cargo-fuzz. Concerning #14, why not rely on https://docs.rs/bitwise/0.1.1/bitwise/ ?

I'll put that in the Cargo.toml so that people in crates.io get the docs of the latest release, but in the github repo I would like to be able to see the docs of the master branch.

For some reason it works fine for the bitintr crate (same token and everything), but for bitwise it fails :/

blt commented 7 years ago

Ah, the reasoning makes sense! Odd that it fails for this project alone.

On Thu, Mar 2, 2017, at 02:56 AM, gnzlbg wrote:

Will do, with regard to cargo-fuzz. Concerning #14[1], why not rely on https://docs.rs/bitwise/0.1.1/bitwise/ ?

I'll put that in the Cargo.toml so that people in crates.io get the docs of the latest release, but in the github repo I would like to be able to see the docs of the master branch. For some reason it works fine for the bitintr crate (same token and everything), but for bitwise it fails :/ — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub[2], or mute the thread[3].

Links:

  1. https://github.com/gnzlbg/bitwise/issues/14
  2. https://github.com/gnzlbg/bitwise/pull/15#issuecomment-283622352
  3. https://github.com/notifications/unsubscribe-auth/AAAN-1O0o403DvjtZxA4BwAGDAw3cH-Wks5rhqBugaJpZM4MMpYb