search

google / fuzzbench

FuzzBench - Fuzzer benchmarking as a service.
https://google.github.io/fuzzbench/
Apache License 2.0
1.11k stars 270 forks source link

SBFT'23 Benchmarks #1841

Open mvanotti opened 1 year ago

mvanotti commented 1 year ago

Will these benchmarks be added to FuzzBench? Seems like they were added and removed at some point.

oliverchang commented 1 year ago

@Alan32Liu @jonathanmetzman thoughts?

DonggeLiu commented 1 year ago

Will these benchmarks be added to FuzzBench? Seems like they were added and removed at some point.

Sorry I missed this:

These benchmarks are available under the SBFT'23 tag. We removed them from the master branch after the competition because running every experiment with them is too time-consuming.

mvanotti commented 1 year ago

Thanks! Would it make sense to keep them in the main branch but not run them or to at least document that for newcomers?

DonggeLiu commented 1 year ago

Thanks! Would it make sense to keep them in the main branch but not run them or to at least document that for newcomers?

May I ask what is the benefit of having them in the master branch? IIUC, that would require extra work in filtering those benchmarks from automatic experiments. Most experiments do not use them anyway.

DonggeLiu commented 1 year ago

I reckon it is a good idea to document that. I can post our paper with the link to SBFT'23 tag in the publication section, once ICSE has a formal link to the paper.