search

thu-ml / tianshou

An elegant PyTorch deep reinforcement learning library.
https://tianshou.org
MIT License
7.79k stars 1.12k forks source link

RNN support for TD3 and SAC #795

Open qtomcatq opened 1 year ago

qtomcatq commented 1 year ago

Is RNN support available for TD3 and SAC algorithms? On the website of Tianshou there is a table that says that RNNs are not supported for both TD3 and SAC, however, there are functions RecurrentCritic and RecurrentActorProb that give RNN support for critic and actor regardless of RL algorithm. The way these functions are supposed to work is exactly what I need for my project, but no idea if they work for TD3 and SAC.

Trinkle23897 commented 1 year ago

It should work if you change ActorProb/Critic to RecurrentActorProb/RecurrentCritic. But as many users reported (#486), it only "works" for the workflow/pipeline perspective instead of ML perspective unfortunately.

qtomcatq commented 1 year ago

Thank you for your response. So, it doesn't work from ML perspective for all algorithms with RNN, not only for TD3 and SAC? If not, do you have any example of working code at least for one algorithm with RNN?

Trinkle23897 commented 1 year ago

https://github.com/thu-ml/tianshou/issues/567#issuecomment-1065786212