Modern library to test & dev ideas on GFlowNets?

[pieris98]

Hey Alex and others, First of all thank you for your generous work in the field and for spreading the ideas with open source code for the rest of us!

I want to experiment with GFlowNets and potentially develop stuff for my Master's thesis. I've stumbled upon this repo, as well as recursion's repo (also seems modern) and the torchgfn repo in GFNOrg (has useful tutorials for the basics and the first handful of loss objectives e.g. FM, DB, TB, SubTB).

I've seen most recent commits here in your repo, and the README mentions development and experimentation as its purpose.

Which one would you suggest is better to use?

Thanks for the help!

[josephdviviano]

IMHO, my (biased as I am a developer for torchgfn) take is the following:

• This project is quite feature complete but heavy. If your project is applied in nature and ideally close in spirit to what has been done already in this project, you should definitely start here and leverage all of the hard work put into this project.
• The recursion repo is quite modern as well but devoted mostly to the area of drug discovery. This isn't totally true, you can do lots of basic research in their repo as well, but figuring out how to do so might be tricky without insider knowledge.
• torchgfn was designed from the beginning to be a modular and lightweight testbed for smaller GFN projects. It still requires a good amount of work to have the same features as either this repo or the recursion repo, which we plan to build over the remainder of the summer. However, it will hopefully get out of your way more often than the others. The flipside is that it currently lacks any meaningful applied environments that you can find in the other repos, and is missing some modern advances in gflownets (which we believe should be easy to add and are planning to do - if you have any interest, you could help).

In the end I think your choice should be guided by the specific kind of project you want to work on. Is the idea to use GFlowNets for a particular purpose, or rather to investigate the properties of GFlowNets on toy environments?

[pieris98]

Hey @josephdviviano , Thanks for your fast reply!

The idea is to eventually in the long run (till October) develop and benchmark GFlowNets in a 3D objectgeneration task/environment (ideally a mesh or a simple manifold).

I suppose this will require Continuous GFlowNets (correct me if I'm wrong), which I've seen are still very complicated and buggy to develop under torchgfn.

Having said that, I'm still learning the very basics and trying to grasp the differences e.g. between FM, DB, TB, Sub-TB, EB-GFN, and what their appropriate applications are.

I'd be glad to contribute whatever's at priority here (feel free to email/PM me), but I'm still lacking deeper knowledge needed for the time being.

Thanks again for your tips!

[josephdviviano]

Yes, you're correct that it's complicated to develop these under torchgfn (imho coninuous GFNs are complicated and hard to scale in general). I'm not sure how much easier it is to develop these under the other frameworks - if you can identify a specific area where it can be improved, please do file an issue under that project.

You might actually want to read this paper before committing to your approach: https://arxiv.org/abs/2402.06121

You should see if this repo could be adapted to your purposes. It surely has handled the largest scale continuous GFN applications to date. I would not recommend the recursion repo for this purpose. Best of luck!

[alexhernandezgarcia]

Hi @pieris98 thanks for your interest in this repo and for getting in touch!

@josephdviviano has summarised the stronger points of torchgfn and Recursion's codebase, so let me tell you a few words about this one, in relation to my understanding of your plans. As a disclaimer, be aware that I have largely contributed to this repository, which will surely add a bias.

I would say that at this point, it is very straightforward for us developers to implement new environments (unless they are overly complicated) by inheriting the existing environments and/or taking the existing code as a template. It will surely not be so straightforward for someone who does not know the code so well, which is why we are currently writing tutorials and documentation. However, it should not be too hard either.

For example, if the problem at hand is an N-dimensional continuous manifold, then as a starting point you could simply create a new class off the ContinuousCube and that would most likely be it. See, for example, the LatticeParameters, whose code is 370 lines, mostly docstring and the methods to handle environment-specific constraints. Actually, if what you want is simply an environment that samples N continuous numbers, you don't even need to create a new class because that is essentially what the Cube does. Assuming you have already implemented your proxy class for the reward function, let's call it myproxy, and that your manifold has 10 dimensions, you can just run:

python main.py env=ccube env.n_dim=10 proxy=myproxy

You can replace myproxy by uniform (a dummy, constant reward, which will make the GFlowNet learn to sample uniformly from the sample space) to see how it runs.

If you want to change things of the Cube environment to fit your needs, then I would recommend implementing a new class, as in the LatticeParameters. If you could still stick to inheriting from the Cube environment, then it should be pretty easy. If you need to implement a continuous environment from scratch, I would not be surprised if it takes some pain, because I have gone through that a couple of times. The good news is that you can 1. ask me for tips and 2. take the Cube and the CTorus code as templates.

I mentioned the proxy too. That is something you will have to implement to fit your needs. It should be super easy. You have to do two things:

1. Implement a proxy class, inheriting from the base Proxy. See some example here: https://github.com/alexhernandezgarcia/gflownet/tree/main/gflownet/proxy
2. Create a YAML config file for your proxy (only necessary if you want to use Hydra and configuration files, as in the example above). See the examples here: https://github.com/alexhernandezgarcia/gflownet/tree/main/config/proxy

I hope this helps!