ejmahler / RustFFT

RustFFT is a high-performance FFT library written in pure Rust.
Apache License 2.0
706 stars 49 forks source link

Add basic fuzzing support for planned FFT, split by instruction set #112

Closed Shnatsel closed 6 months ago

Shnatsel commented 1 year ago

This will only cover the combinations of the input and algorithm that the planner selects. It also doesn't perform correctness checks, only checking for memory safety issues.

On the upside this is easy and straightforward to implement, and doesn't result in any false positives due to float precision issues.

Shnatsel commented 1 year ago

The AVX fuzz target is slow if run with both debug assertions and address sanitizer. You have to use either cargo +nightly fuzz --release avx or cargo +nightly fuzz --sanitizer=none avx

--release is preferred because ASAN catches more severe issues, so it's best to keep it on.

HEnquist commented 6 months ago

I had forgotten about this and just rediscovered it. I can't help wondering what this tests that isn't already covered by the existing tests. All algorithms already have tests that compare their output with the reference FFT. The FFT algorithms also don't depend on the input values, the code path is the same no matter what values are in the input. The only important parameter is the length, which is used to select algorithm. Once that is done, the code path is completely fixed. As an example, a length-2 FFT is [a, b] ==> [a+b, a-b]. Longer lengths are of course use longer expressions, but they are equally fixed.

To me it would make a lot more sense to test what happens if you feed the FFT various combinations of invalid buffers. That should return errors, and never panic.

Shnatsel commented 6 months ago

I wrote this before I actually dove into the implementations of FFT. I tend to agree that if the execution path isn't dependent on the data, there isn't a whole lot for the fuzzer to test. Running those tests under Address Sanitizer would probably achieve similar results. So I'm going to go ahead and close this.