search

ShengranHu / ADAS

Automated Design of Agentic Systems
Apache License 2.0
1.02k stars 148 forks source link

Why `bootstrap confidence interval` instead of direct use of `accuracy`? #15

Open yqy2001 opened 2 months ago

yqy2001 commented 2 months ago

Hello~

Why in the code, the optimization objective is bootstrap confidence interval, rather than directly using the accuracy?

Thank you.

ShengranHu commented 2 months ago

Hello, thanks for your questions. I anticipate that it may not be significantly different to use accuracy. We use median and confidence interval to provide more information to the meta agent (and match the final reported metrics).