search

RatInABox-Lab / RatInABox

A python package for modelling locomotion in complex environments and spatially/velocity selective cell activity.
MIT License
176 stars 31 forks source link

Actor critic system functional draft #7

Closed danielmk closed 1 year ago

danielmk commented 1 year ago

This is a draft of the actor critic implementation.

Both the Critic code as well as the demo are rough but they are functional see the below output (single trajectory). critic_demo_output

The neuron implementations in Critic.py could also be part of Neurons.py I don't feel strongly about separating them. Let me know what you think.

TomGeorge1234 commented 1 year ago

Awesome, I took a cursory look and it looks great. We can discuss over the call next Friday but my feeling is that these two new files should live in contribs, perhaps in their own folder "ActorCritic" or something like that. Thanks for contributing this though, its really exciting!

danielmk commented 1 year ago

I'm gonna close this for the moment as it needs a lot more work and reopen when I have it properly factored and put into contribs etc.

TomGeorge1234 commented 1 year ago

@danielmkjust in case you're still working on this stuff (or for anyone else who finds their way here) I wanted to make you aware of #30 . In summary progress is being made to wrap RiaB inside the gymnasium framework so RL can be done under openAIs framework (and the ecosystem that comes with) but powered by RiaB. Worth being aware.

danielmk commented 1 year ago

Thank you for letting me know @TomGeorge1234 Worth being aware indeed. I prioritized other projects for the moment but hope to get back to the reinforcement learning work.