Closed saleml closed 5 months ago
Hmm, on my machine, the training isn't perfect (the default options undertrain the policy) but I definitely sample from both modes.
Were your experiments off of master
or is this maybe related to changes we made RE: off policy training in the other open pr (https://github.com/GFNOrg/torchgfn/pull/174)?
I'm investigating the issue. Indeed, different behavior on mater
and on fix_off_policy
.
One thing worth noting is that with the original number of trajectories (1.28e6), sure the samples are from one mode only, but are more accurate.
Edit1: And when running the code for slightly longer on the fix_off_policy
branch (3e6 trajectories), I obtain a figure that is similar to the one obtained with master
.
In fix_off_policy
, I had forgotten a keyword. The problem is fixed in https://github.com/GFNOrg/torchgfn/pull/174/commits/89c72b5add431fcd8a787323cdc14aee7ee1ffe8
I likely need to change the default options to enable off policy exploration