Episodic training with TQC?

Stable-Baselines-Team / stable-baselines3-contrib

Contrib package for Stable-Baselines3 - Experimental reinforcement learning (RL) code

https://sb3-contrib.readthedocs.io

MIT License

442 stars 166 forks source link

Closed Armandpl closed 5 months ago

Armandpl commented 5 months ago

https://github.com/Stable-Baselines-Team/stable-baselines3-contrib/blob/f8aa42898503d4536856595b186dddbd4bc6f9b9/sb3_contrib/tqc/tqc.py#L91 Any reason the typing prevents passing a tuple with a number of episodes like with SAC in sb3? I'd like to use TQC to control a real robot but the control freq while training is too slow on my computer, so I need to train every episode instead of every step.

araffin commented 5 months ago

Hello, no particular reason, we probably forgot to update the type hint, I would appreciate a PR =)

o I need to train every episode instead of every step.

You might be interested in:

async train (I did it with TQC in the past): https://github.com/DLR-RM/stable-baselines3/issues/715#issuecomment-1003696395
SBX (my go to now), up to 20x faster than the pytorch version with the right parameters: https://github.com/araffin/sbx

EDIT: QR-DQN need to be updated too

Armandpl commented 5 months ago

thanks, I will check out SBX :) I just submitted a PR to update the typing