In some rare cases, (encountered once) noise sampling in gSDE can break. It happened once with TQC on HalfCheetahBulletEnv-v0, after 800k timesteps. For some reason, the entropy loss diverged. Might be related to https://github.com/DLR-RM/rl-baselines3-zoo/issues/322
Description
In some rare cases, (encountered once) noise sampling in gSDE can break. It happened once with TQC on HalfCheetahBulletEnv-v0, after 800k timesteps. For some reason, the entropy loss diverged. Might be related to https://github.com/DLR-RM/rl-baselines3-zoo/issues/322
Run detailed here: https://wandb.ai/openrlbenchmark/sb3/runs/27cez5ua
To reproduce:
System Info