neulab / ExplainaBoard

Interpretable Evaluation for AI Systems
MIT License
359 stars 36 forks source link

Update condition to apply Student's t-test #558

Closed tetsuok closed 1 year ago

tetsuok commented 1 year ago

Overview

This PR updates the condition when calc_confidence_interval applies Student's t-test.

Details

As per our discussion in #509, the condition to perform t-test is added: when the sample size is large enough. As per this change, some tests in metric_test.py were required to update so that calc_confidence_interval performs t-test in the tests rather than bootstrapping.

References

Blocked by