eval_env and train_env are not of the same type

[KantaphatLeelakunwet]

Hi @ScheiklP, I would like to get some results from the evaluation during training, so I added the EvalCallback to the callback_list in sofa_zoo/common/sb3_setup.py. I also created another evaluation environment called eval_env by directly assigning env. (I have tried copy.deepcopy but it throws max depth recursion error.) A few modifications have been made to lines 150-160.

However, the type error occur. The error message is below.

/bd_targaryen/users/kleelakunwet/miniconda3/envs/sofa/lib/python3.9/site-packages/stable_baselines3/common/callbacks.py:414: UserWarning: Training and eval env are not of the same type<stable_baselines3.common.vec_env.vec_transpose.VecTransposeImage object at 0x7fb423fae220> != <stable_baselines3.common.vec_env.vec_frame_stack.VecFrameStack object at 0x7fb423f75a60>
  warnings.warn("Training and eval env are not of the same type" f"{self.training_env} != {self.eval_env}")
[ERROR]   [PythonScript] AssertionError: Error while synchronizing normalization stats: expected the eval env to be a VecNormalize but got <stable_baselines3.common.vec_env.vec_monitor.VecMonitor object at 0x7fb423f95fa0> instead. This is probably due to the training env not being wrapped the same way as the evaluation env. Training env type: <stable_baselines3.common.vec_env.vec_normalize.VecNormalize object at 0x7fb423f7b130>.
  File "/bd_targaryen/users/kleelakunwet/sofa_zoo/sofa_zoo/envs/deflect_spheres/ppo.py", line 107, in <module>
    model.learn(
  File "/bd_targaryen/users/kleelakunwet/miniconda3/envs/sofa/lib/python3.9/site-packages/stable_baselines3/ppo/ppo.py", line 315, in learn
    return super().learn(
  File "/bd_targaryen/users/kleelakunwet/miniconda3/envs/sofa/lib/python3.9/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 277, in learn
    continue_training = self.collect_rollouts(self.env, callback, self.rollout_buffer, n_rollout_steps=self.n_steps)
  File "/bd_targaryen/users/kleelakunwet/miniconda3/envs/sofa/lib/python3.9/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 200, in collect_rollouts
    if not callback.on_step():
  File "/bd_targaryen/users/kleelakunwet/miniconda3/envs/sofa/lib/python3.9/site-packages/stable_baselines3/common/callbacks.py", line 114, in on_step
    return self._on_step()
  File "/bd_targaryen/users/kleelakunwet/miniconda3/envs/sofa/lib/python3.9/site-packages/stable_baselines3/common/callbacks.py", line 219, in _on_step
    continue_training = callback.on_step() and continue_training
  File "/bd_targaryen/users/kleelakunwet/miniconda3/envs/sofa/lib/python3.9/site-packages/stable_baselines3/common/callbacks.py", line 114, in on_step
    return self._on_step()
  File "/bd_targaryen/users/kleelakunwet/miniconda3/envs/sofa/lib/python3.9/site-packages/stable_baselines3/common/callbacks.py", line 449, in _on_step
    sync_envs_normalization(self.training_env, self.eval_env)
  File "/bd_targaryen/users/kleelakunwet/miniconda3/envs/sofa/lib/python3.9/site-packages/stable_baselines3/common/vec_env/__init__.py", line 73, in sync_envs_normalization
    assert isinstance(eval_env_tmp, VecNormalize), (

Do you know what might be the problem here? Thank you in advance!

[ScheiklP]

Hi @KantaphatLeelakunwet , well the error says that both eval and train env should be of the same type, and that they are not. You should probably create two independent vectorized envs. Both in the same way.

However, I have not used a separate eval env yet. Maybe the people over at the stablebaselines3 repository are the better people to ask.