Closed jonastim closed 9 months ago
All modified and coverable lines are covered by tests :white_check_mark:
Comparison is base (
7499d8b
) 99.90% compared to head (3c0686b
) 99.90%.:exclamation: Current head 3c0686b differs from pull request most recent head dd3b34b. Consider uploading reports for the commit dd3b34b to get more accurate results
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
I can't seem to re-run tests. Looks like these unrelated tests should be less sensitive
======================================================================
FAIL: test_DM (coba.tests.test_environments_filters.OpeRewards_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/runner/work/coba/coba/coba/tests/test_environments_filters.py", line 2390, in test_DM
self.assertAlmostEqual(new_interactions[0]['rewards'].eval('c'),.79699, places=4)
AssertionError: 0.7970473766326904 != 0.79699 within 4 places (5.7376632690453455e-05 difference)
======================================================================
FAIL: test_DM_action_not_hashable (coba.tests.test_environments_filters.OpeRewards_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/runner/work/coba/coba/coba/tests/test_environments_filters.py", line 2404, in test_DM_action_not_hashable
self.assertAlmostEqual(new_interactions[0]['rewards'].eval(['c']),.79699, places=4)
AssertionError: 0.7970473766326904 != 0.79699 within 4 places (5.7376632690453455e-05 difference)
----------------------------------------------------------------------
I can't seem to re-run tests. Looks like these unrelated tests should be less sensitive
====================================================================== FAIL: test_DM (coba.tests.test_environments_filters.OpeRewards_Tests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/runner/work/coba/coba/coba/tests/test_environments_filters.py", line 2390, in test_DM self.assertAlmostEqual(new_interactions[0]['rewards'].eval('c'),.79699, places=4) AssertionError: 0.7970473766326904 != 0.79699 within 4 places (5.7376632690453455e-05 difference) ====================================================================== FAIL: test_DM_action_not_hashable (coba.tests.test_environments_filters.OpeRewards_Tests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/runner/work/coba/coba/coba/tests/test_environments_filters.py", line 2404, in test_DM_action_not_hashable self.assertAlmostEqual(new_interactions[0]['rewards'].eval(['c']),.79699, places=4) AssertionError: 0.7970473766326904 != 0.79699 within 4 places (5.7376632690453455e-05 difference) ----------------------------------------------------------------------
Totally agree. Are these the only tests you're having problems with? I've tried to bump down a lot of tests over time.
The main change is for the off-policy evaluator to log the action and probability of the learning model rather than that of the logged data (which is identical for all models and not very useful when trying to compare different learners).
Introduced a helper function for sampling the actions and used if in some other code placed to avoid redundancy.