This PR adds seed argument to Metric class to fix #537.
Details
Metric's calc_confidence_interval method uses random generator for bootstrapping, but we have no way to provide a seed to the method to make the behavior reproducible. To address this issue, adds seed argument to Metric class, as
described in https://github.com/neulab/ExplainaBoard/issues/537#issuecomment-1274340748.
As per this change, reverted a hot fix in #550. This is to use almost equal assertions to assert confidence intervals computed by bootstrapping.
Overview
This PR adds
seedargument toMetricclass to fix #537.Details
Metric'scalc_confidence_intervalmethod uses random generator for bootstrapping, but we have no way to provide a seed to the method to make the behavior reproducible. To address this issue, addsseedargument toMetricclass, as described in https://github.com/neulab/ExplainaBoard/issues/537#issuecomment-1274340748.As per this change, reverted a hot fix in #550. This is to use almost equal assertions to assert confidence intervals computed by bootstrapping.
References
537
Blocked by