I have noticed that whenever, evalution run is executed, in the successive training log, the mean episode length becomes greater than my set episode horizon. So, I guess the agent doesnt continue the unfinished training rollout ( the one before the evaluation was triggered), rather the agent resets and runs a new rollout episode but it still somehow counts the time steps from its unfinished episode as a part of the new rollout episode.
In the logs which I have attached, I set the horizon to 15. And it can be seen that the rollout/ep_len_mean changes from 15 to 15.4 after evaluation.
🐛 Bug
I have noticed that whenever, evalution run is executed, in the successive training log, the mean episode length becomes greater than my set episode horizon. So, I guess the agent doesnt continue the unfinished training rollout ( the one before the evaluation was triggered), rather the agent resets and runs a new rollout episode but it still somehow counts the time steps from its unfinished episode as a part of the new rollout episode.
In the logs which I have attached, I set the horizon to 15. And it can be seen that the rollout/ep_len_mean changes from 15 to 15.4 after evaluation.
Relevant log output / Error message
System Info
No response
Checklist