google / fuzzbench

FuzzBench - Fuzzer benchmarking as a service.
https://google.github.io/fuzzbench/
Apache License 2.0
1.11k stars 269 forks source link

freetype_ftfuzzer oss-fuzz benchmark uses outdated commit #1925

Open ndrewh opened 11 months ago

ndrewh commented 11 months ago

the benchmark.yaml in master currently is quite misleading:

https://github.com/google/fuzzbench/blob/c734a74ba59deada3cf4d7e37009125e26d51701/benchmarks/freetype2_ftfuzzer/benchmark.yaml#L15-L18

That commit (cd02d359a6d0455e9d16b87bf9665961c4699538) is not from 2023, it is from 2017. Imagine my surprise when a long fuzzing run ended and I look through the coverage report and find out I'm looking at 6-year-old code!

As far as the freetype build issue referenced in #1688 goes, it looks like someone is not setting or using SANITIZER correctly. If you add export SANITIZER=none to the build.sh script that is copied in when you run:

python3 benchmarks/oss_fuzz_benchmark_integration.py -p freetype2 -f ftfuzzer -n ftfuzzer_01f23 -c 01f23c39e01d2c739fd8abc69aff9dbf13b46202 -d 2023-12-02T04:13:28+00:00

it builds fine. I didn't look into it further, and this probably isn't an ideal solution since presumably some fuzzers want sanitizers in some contexts.