Closed DonggeLiu closed 1 week ago
- Update benchmarks using the new oracles: optimal-targets, all-public-candidates.
I understood it as you'd like to also add new benchmarks? At the moment it's only removing
all
Yep, I need to confirm which oracles to use before generate new benchmarks : )
Yep, I need to confirm which oracles to use before generate new benchmarks : )
Do: far-reach-but-low-coverage
, optimal-targets
, easy-params-far-reach
/gcbrun exp -n dg
Looking good: https://llm-exp.oss-fuzz.com/Result-reports/scheduled/2024-07-06-weekly-all/
Will select a new comparison set for daily improvement/regression monitoring.
/gcbrun exp -n dg
The new comparison
set is selected from the 07-06 experiment, containing benchmarks with:
/gcbrun skip
optimal-targets
,all-public-candidates
.@DavidKorczynski It appears that
optimal-targets
does not work well with many benchmarks. E.g., empty list forlibxml2
, 'Project not in the database' foravahi
. Is this expected?