SBFT'24 Competition - No mutation testing - Githubissues

google / fuzzbench

FuzzBench - Fuzzer benchmarking as a service.

https://google.github.io/fuzzbench/

Apache License 2.0

1.09k stars 266 forks source link

SBFT'24 Competition - No mutation testing #1946

Open phi-go opened 8 months ago

phi-go commented 8 months ago

@Alan32Liu could you try another coverage experiment with this branch?

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name sbft-standard-cov-01-17 --fuzzers libafl libfuzzer mystique pastis bandfuzz tunefuzz fox

DonggeLiu commented 8 months ago

Oh scratch my comment in the other PR. Will run this now

@Alan32Liu could you try another coverage experiment with this branch?

DonggeLiu commented 8 months ago

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name sbft-standard-cov-01-17 --fuzzers libafl libfuzzer mystique pastis bandfuzz tunefuzz fox

DonggeLiu commented 8 months ago

ops, did you forget to make the dummy change to trigger exp? : ) Unfortunately I don't have the permission to add it to your branch.

DonggeLiu commented 8 months ago

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name sbft-standard-cov-01-17 --fuzzers libafl libfuzzer mystique pastis bandfuzz tunefuzz fox

phi-go commented 8 months ago

@Alan32Liu, done, it was in the sbft branch that I removed ...

It also seems that we do not have a report for pastis, so we could skip that fuzzer to save some time in the eval.

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name sbft-standard-cov-01-18 --fuzzers libafl libfuzzer mystique bandfuzz tunefuzz fox

DonggeLiu commented 8 months ago

Experiment sbft-standard-cov-01-17 data and results will be available later at: The experiment data. The experiment report.

phi-go commented 8 months ago

Ah you were faster, how long would you expect this experiment to take?

DonggeLiu commented 8 months ago

Ah you were faster, how long would you expect this experiment to take?

1-2 days? I will need to re-run the command because the last one was somehow interrupted.

DonggeLiu commented 8 months ago

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name sbft-standard-cov-01-18 --fuzzers libafl libfuzzer mystique bandfuzz tunefuzz fox

phi-go commented 8 months ago

Oh, in that case let's include pastis. Sorry for the back and forth.

phi-go commented 8 months ago

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name sbft-standard-cov-01-18 --fuzzers libafl libfuzzer mystique pastis bandfuzz tunefuzz fox

DonggeLiu commented 8 months ago

Experiment sbft-standard-cov-01-18 data and results will be available later at: The experiment data. The experiment report.

DonggeLiu commented 8 months ago

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name sbft-standard-cov-01-18-p --fuzzers libafl libfuzzer mystique pastis bandfuzz tunefuzz fox

phi-go commented 8 months ago

Thank you!

DonggeLiu commented 8 months ago

Experiment sbft-standard-cov-01-18-p data and results will be available later at: The experiment data. The experiment report.

phi-go commented 8 months ago

@Alan32Liu we had one final commit... Could you start the experiment again? Also did something go wrong with the experiment, I don't see any build directories?

DonggeLiu commented 8 months ago

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name sbft-standard-cov-01-19-p --fuzzers libafl libfuzzer mystique pastis bandfuzz tunefuzz fox

phi-go commented 8 months ago

@Alan32Liu can you post the link to the results of the experiment as well? 🙂

DonggeLiu commented 8 months ago

This is so strange: The experiment was not requested.

This also happened on the previous 2 exps. Let me try again.

DonggeLiu commented 8 months ago

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name sbft-standard-cov-01-20 --fuzzers libafl libfuzzer mystique pastis bandfuzz tunefuzz fox

DonggeLiu commented 8 months ago

Pretty sure it is working this time!

Experiment sbft-standard-cov-01-20 data and results will be available later at: The experiment data. The experiment report. OR The experiment report.