VowpalWabbit / estimators

Estimators to perform off-policy evaluation
BSD 3-Clause "New" or "Revised" License
13 stars 17 forks source link

handle empty candidates #70

Closed orenmichaely closed 1 year ago

orenmichaely commented 1 year ago

In evaluations we hit an edge case case where the candidates list is empty when getting the Cressie Read CI.

By default wmin is set to 0 and wmax to inf, so I think we are hitting an edge case where the two conditions are not satisfied

if isclose(y z, 0, abs_tol=atol atol): gstar = x - sqrt(2) atol candidates.append(gstar) elif z <= 0 and y z >= 0: gstar = x - sqrt(2 y z) candidates.append(gstar)

Based on the c++ code, rmin is the best way to deal with empty candidates.

https://github.com/VowpalWabbit/vowpal_wabbit/blob/748bc56735a000927f7795e5f316ca3861302a1d/vowpalwabbit/core/src/distributionally_robust.cc#L236