search

nnaisense / evotorch

Advanced evolutionary computation library built directly on top of PyTorch, created at NNAISENSE.
https://evotorch.ai
Apache License 2.0
1.01k stars 63 forks source link

Question about ES-HyperNEAT integration #2

Closed pablogranolabar closed 2 years ago

pablogranolabar commented 2 years ago

Hi! Neat project, thanks for making it available. I've been experimenting with ES-HyperNEAT using Gym wrappers. I am curious if an ES-HyperNEAT custom Gym implementation with SteadyStateGA would be feasible and parallelize'able to GPU?

NaturalGradient commented 2 years ago

Hi @pablogranolabar ! This is the kind of stuff we have built EvoTorch for :) ! We think it should be possible to implement HyperNEAT (and therefore ES-HyperNEAT) in EvoTorch, but we don't have an implementation ourselves. I think you should start with subclassing NEProblem to build a tool for mapping ES-HyperNEAT solutions (e.g. the decoding component) to torch modules. Regarding the HyperNEAT component, you'll want to either inherit from SearchAlgorithm or SteadyStateGA to build that kind of functionality. As a starting point, take a look at our tutorial on building custom SearchAlgorithm instances: https://docs.evotorch.ai/v0.1.1/advanced_usage/custom_ea/.

Regarding parallelization support for Gym, we're planning to explore this in a future release, but we currently only support the non-vectorized version of Gym. But as we have scaling with Ray, this shouldn't stop you from making progress. I will say that SteadyStateGA itself is vectorized + does support GPU, so you can use that as a starting point.

If you make some progress implementing this, please let us know!

NaturalGradient commented 2 years ago

Hi @pablogranolabar. We're closing this issue for now, but feel free to open a pull request if you making some progress in implementing this algorithm!