search

GFNOrg / torchgfn

GFlowNet library
https://torchgfn.readthedocs.io/en/latest/
Other
209 stars 26 forks source link

`test_scripts` failing by small deltas #138

Closed josephdviviano closed 9 months ago

josephdviviano commented 11 months ago 
=============================================== short test summary info ===============================================
FAILED test_scripts.py::test_hypergrid[8-2] - assert 0.0008832142921164632 < 0.00073
FAILED test_scripts.py::test_hypergrid[16-4] - assert 2.456717993482016e-05 < 2.45e-05
FAILED test_scripts.py::test_discreteebm[0.1-2] - assert 0.002971753478050232 < 0.0026
================================ 3 failed, 9 passed, 2 warnings in 1225.99s (0:20:25) =================================

I'm not sure if we should actually worry about this (or rather whether this is just due to the seed not being applied to enough RNGs in the scripts -- i.e., we don't seed numpy or random).

saleml commented 11 months ago

Not really harmful. Discrepancies in gpu/cpu/OS I believe. The scripts still run fine on my laptop actually.

To avoid failure, let's add more seeds, and a set a larger epsilon for the gap between obtained result and expected result ?

josephdviviano commented 9 months ago

Seeds added and tolerances increased in https://github.com/GFNOrg/torchgfn/pull/147