I saw this reply on a similar problem I had with the render_all() method. Though in my case I am using a VecNormalize() wrapper around my DummyVecEnv. In the solution quoted a DummyVecEnv was made that was used for training, and then another env was instantiated for the prediction/testing that could be used with render all. In my case this won't work since I need the VecNormalize to normalize observations and reward.
env = make_vec_env(env_maker, n_envs=1, monitor_dir=log_dir)
env = VecNormalize(env, norm_obs=True, norm_reward=True, clip_obs=10.)
model = PPO2('MlpLstmPolicy', env, verbose=1, nminibatches=1, policy_kwargs=policy_kwargs,)
callback = SaveOnBestTrainingRewardCallback(check_freq=1000, log_dir=log_dir, env=env, verbose=1)
# model = PPO2('MlpLstmPolicy', env, verbose=1)
model.learn(total_timesteps=5000, callback=callback, log_interval=10)
env.norm_reward = False
env.training = False
mean_reward, std_reward = evaluate_policy(model, env, n_eval_episodes=1)
print(f"mean_reward:{mean_reward:.2f} +/- {std_reward:.2f}")
# I get the expected reward here using evaluate_policy()
plt.figure(figsize=(15,6))
plt.cla()
env.render_all()
plt.show()
# This part doesn't work because of the same error
What can I do to use render_all() method (Or any other attribute like env.history for that matter) while maintaining the VecNormalize() environment?
The problem is something inside the
DummyVecEnv
which resets the environment automatically after it is done.Also, there was a mistake in your code. Try this:
Originally posted by @AminHP in https://github.com/AminHP/gym-anytrading/issues/1#issuecomment-580320757
I saw this reply on a similar problem I had with the render_all() method. Though in my case I am using a VecNormalize() wrapper around my DummyVecEnv. In the solution quoted a DummyVecEnv was made that was used for training, and then another env was instantiated for the prediction/testing that could be used with render all. In my case this won't work since I need the VecNormalize to normalize observations and reward.
What can I do to use render_all() method (Or any other attribute like env.history for that matter) while maintaining the VecNormalize() environment?