search

facebookresearch / Pearl

A Production-ready Reinforcement Learning AI Agent Library brought by the Applied Reinforcement Learning team at Meta.
MIT License
2.68k stars 166 forks source link

Simplify actor-critic with shared layers #73

Open panoskyriakis opened 8 months ago

panoskyriakis commented 8 months ago

The current implementation of ActorCriticBase makes it a bit trick to have custom actor and critic networks that have shared layers. This is because the instantiation of the networks happen in the ActorCriticBase class. You can probably get around it but it's very tricky. I'd recommend you pass in the actor/critic objects rather than the class types and let the user initialize them.

jb3618columbia commented 8 months ago

Thank you for the comment. Yes, we have that planned for actor-critic methods. Note that for value based methods, we do give the option to users to pass in a network instance.

jb3618columbia commented 7 months ago

Hi,

Just following up on this. We have updated the actor-critic base class to also be able to pass in actor and critic networks as arguments. Thank you for this suggestion. Please let us know if you notice any errors when using it. It will be helpful for us.