[BUG] Env does not reset when it's terminated

[modanesh]

Problem Description

Checklist

• [X] I have installed dependencies via poetry install (see CleanRL's installation guideline.
• [X] I have checked that there is no similar issue in the repo.
• [X] I have checked the documentation site and found not relevant information in GitHub issues.

Current Behavior

Whenever the environment is terminated or truncated, it should be reset, but that doesn't happen in a few implementations. For example, in this for loop (https://github.com/vwxyzjn/cleanrl/blob/master/cleanrl/sac_continuous_action.py#L221), we should check whether the envs is terminated/truncated or not, and reset it accordingly.

[Howuhh]

These are vectorized environments with the help of gym.vector.SyncVectorEnv. They will auto-reset on truncation or termination. This behaviour is described in Gymnasium documentation.

[vwxyzjn]

Yeah see "To prevent terminated environments waiting until all sub-environments have terminated or truncated, the vector environments autoreset sub-environments after they terminate or truncated. As a result, the final step’s observation and info are overwritten by the reset’s observation and info."

Closing the issue then since this is a non-issue.