data recording and saving method

[Xiong5Heng]

• [x] I have marked all applicable categories:
  • [x] exception-raising bug
  • [x] RL algorithm bug
  • [x] documentation request (i.e. "X is missing from the documentation.")
  • [x] new feature request
  • [x] design request (i.e. "X should be changed to Y.")
• [x] I have visited the source website
• [x] I have searched through the issue tracker for duplicates
• [x] I have mentioned version numbers, operating system and environment, where applicable:

  import tianshou, gymnasium as gym, torch, numpy, sys
  print(tianshou.__version__, gym.__version__, torch.__version__, numpy.__version__, sys.version, sys.platform)

Hi,

When I use SubprocVectorEnv, I want to record the rewards from all environments. Do you have similar function just like VecMonitor in SB3 (https://stable-baselines3.readthedocs.io/en/master/guide/vec_envs.html#stable_baselines3.common.vec_env.VecMonitor)?

[MischaPanch]

I would suggest using an environment wrapper for that. At the moment tianshou is primarily an algorithm library, not focused on wrappers. In fact, you could just use the wrapper from SB3 together with tianshou.

Let me know if this answers your question

[Xiong5Heng]

Hi, Thanks for your reply, and I will try your solution. But, if I do not use the wrapper from SB3, are there any other ways to record the rewards from all vector environments?

[MischaPanch]

The best way would be to use an env wrapper. Note that in all examples you can create your own env factory with your own wrapper. I'll try to add a tutorial on how to do that soon.

Apart from that, you can probably use a custom logger. You can also access the buffer directly during training through the trainer, all rewards are saved there.

In very near future we will add support for callbacks during training, which then would provide the simplest way for saving custom data (see #977 #895)

[Xiong5Heng]

The best way would be to use an env wrapper. Note that in all examples you can create your own env factory with your own wrapper. I'll try to add a tutorial on how to do that soon.

Apart from that, you can probably use a custom logger. You can also access the buffer directly during training through the trainer, all rewards are saved there.

In very near future we will add support for callbacks during training, which then would provide the simplest way for saving custom data (see #977 #895)

Thanks for your brilliant work!