search

google / fuzzbench

FuzzBench - Fuzzer benchmarking as a service.
https://google.github.io/fuzzbench/
Apache License 2.0
1.11k stars 269 forks source link

Update libafl-based fuzzers (from AFL++ fork) #2027

Closed addisoncrump closed 3 months ago

addisoncrump commented 3 months ago

Changing forks so @tokatoka can collab with me on this. Supercedes #2021.

addisoncrump commented 3 months ago

@DonggeLiu When possible, could you please run a short experiment to see if the CI is spuriously failing? I think this will still be affected by #2023.

tokatoka commented 3 months ago

are you sure 799c634fef047d3e98355fe1ad17c5226c901a57 is the right commit?

addisoncrump commented 3 months ago

That was the state of main when I checked it... I also tested this locally.

tokatoka commented 3 months ago

and it looks like the problem is from libafl_text we should drop it

addisoncrump commented 3 months ago

Didn't know if we still cared about libafl_text :sweat_smile: I can delete.

DonggeLiu commented 3 months ago

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-08-15-2027-1 --fuzzers libafl libafl_libfuzzer

DonggeLiu commented 3 months ago

Experiment 2024-08-15-2027-1 data and results will be available later at: The experiment data. The experiment report. The experiment report(experimental).

tokatoka commented 3 months ago

idk we only got data for two of the fuzzers?

tokatoka commented 3 months ago

wait but they do have the result for the 30 minutes run in the experiment-data, so everything is working i guess

tokatoka commented 3 months ago

so it's ready to merge?

addisoncrump commented 3 months ago

Somehow the php bug benchmark has bitrotted during this PR!

I'll revert the experiment-relevant changes and then yes, it'll be good to go.

DonggeLiu commented 3 months ago

Strangely FuzzBench reports libafl failed to build with some benchmarks, but the build log says otherwise: image But I don't think this relates to this PR, so let's merge.

Similarly, there are some coverage run failures, but it should not block this PR either: image