search

toybox-rs / Toybox

The Machine Learning Toybox for testing the behavior of autonomous agents.
http://toybox.rs
27 stars 12 forks source link

flaky test in eq #184

Closed jjfiv closed 3 years ago

jjfiv commented 3 years ago

@etosch smells like floating point error?

FAIL: test_prob_eq_compound_manipulation (test.interventions.test_equality.BreakoutEquality)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/Toybox/Toybox/test/interventions/test_equality.py", line 100, in test_prob_eq_compound_manipulation
    self.assertNotEqual(cmp1.differ, cmp2.differ)
AssertionError: ('paddle.velocity.x', 0.0, 12.0) == ('paddle.velocity.x', 0.0, 12.0)
kclary commented 3 years ago

Welp, rerunning removes the earlier failed run information I guess. Won't rerun tests without recording the relevant error logs going forward.

It passed on a rerun I triggered (so: still flaky). The logfile listed the error as an assertion failure due to inequality in game_over