search

FluxML / FluxML-Community-Call-Minutes

The FluxML Community Team repo
50 stars 4 forks source link

Add RL models #4

Closed darsnack closed 3 years ago

darsnack commented 4 years ago

Add standard RL models.

ToucheSir commented 4 years ago

Prior art: https://github.com/JuliaReinforcementLearning/ReinforcementLearningZoo.jl, https://github.com/mkschleg/MinimalRLCore.jl, https://github.com/JuliaML/Reinforce.jl

darsnack commented 4 years ago

Yeah this is a tricky one. I personally find that JuliaReinforcementLearning/* over-uses type abstraction and for my research would just be too confusing to interface with. JuliaML/Reinforce.jl sadly needs some love and a little outdated. I did not know about MinimalRLCore.jl, but I like the implementation and goals of the project so far. Looks promising.

My vote is for another base entirely: https://github.com/JuliaPOMDP/RLInterface.jl. I think the JuliaPOMDP org is very active and doing some very cool things. I like that they made the RL interface very small. One advantage to their approach is that we can take advantage of all the work they've done on simulators. For running a fixed number of experiments of an agent in a for-loop, many of these interfaces are fine. But if you want to run more complex experiments, JuliaPOMDP has a variety of simulators already defined depending on what results you wish to examine.

They already have DQN-style networks implemented. We also have FluxML/Gym.jl where I have suggested that the JuliaPOMDP interface be used.

ToucheSir commented 4 years ago

Looks like there's some standardization happening in the ecosystem as well: https://discourse.julialang.org/t/common-rl-environment-interface/41460

darsnack commented 4 years ago

I am thinking of closing this issue. It doesn't really make sense for RL models to be part of FluxModels.jl. It is rare that you would need an RL model and not an RL environment. So it makes more sense for all things RL to be provided by an RL-specific package.

darsnack commented 3 years ago

I am closing this since we are not focused on RL (for now).