jharwell / sierra

Automation framework for the scientific method in AI research
MIT License
18 stars 1 forks source link

enh/302/vargha-delaney-test #302

Open jharwell opened 2 years ago

jharwell commented 2 years ago

This test can be used to see if you've done enough runs to mitigate uncertainty/noise in your output variables of interested. Thus, adding automatic generation of a graph with {2,3,4,...,N} runs worth of data for a given variable would be really helpful for people using SIERRA, so they know if they need to tell it to do more experimental runs, or not.