AssertionError: Cannot interact with environment 7 which is stepping now.

[ ] I have marked all applicable categories:
- [x] exception-raising bug
- [ ] RL algorithm bug
- [ ] documentation request (i.e. "X is missing from the documentation.")
- [ ] new feature request
[ ] I have visited the source website
[ ] I have searched through the issue tracker for duplicates

[x] I have mentioned version numbers, operating system and environment, where applicable:

import tianshou, gym, torch, numpy, sys
print(tianshou.__version__, gym.__version__, torch.__version__, numpy.__version__, sys.version, sys.platform)
0.4.9 0.25.2 1.12.1+cu116 1.20.3 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)] win32

In each training, after the end of the second epoch, the following error occurs when testing, how can I solve it?

Traceback (most recent call last):
  File "D:\Tony\Documents\yunpan\invest\2022\Quant\code\myclasses\factor\myfactor.py", line 745, in wrapper
    func(*args, **kw)
  File "D:\Tony\Documents\yunpan\invest\2022\Quant\code\myclasses\Securities_trading\SacAgent2.py", line 334, in train
    _result = offpolicy_trainer(
  File "D:\Anaconda3\lib\site-packages\tianshou\trainer\offpolicy.py", line 133, in offpolicy_trainer
    return OffpolicyTrainer(*args, **kwargs).run()
  File "D:\Anaconda3\lib\site-packages\tianshou\trainer\base.py", line 455, in run
    deque(self, maxlen=0)  # feed the entire iterator into a zero-length deque
  File "D:\Anaconda3\lib\site-packages\tianshou\trainer\base.py", line 317, in __next__
    test_stat, self.stop_fn_flag = self.test_step()
  File "D:\Anaconda3\lib\site-packages\tianshou\trainer\base.py", line 346, in test_step
    test_result = test_episode(
  File "D:\Anaconda3\lib\site-packages\tianshou\trainer\utils.py", line 22, in test_episode
    collector.reset_env()
  File "D:\Anaconda3\lib\site-packages\tianshou\data\collector.py", line 427, in reset_env
    super().reset_env(gym_reset_kwargs)
  File "D:\Anaconda3\lib\site-packages\tianshou\data\collector.py", line 136, in reset_env
    rval = self.env.reset(**gym_reset_kwargs)
  File "D:\Anaconda3\lib\site-packages\tianshou\env\venvs.py", line 198, in reset
    self._assert_id(id)
  File "D:\Anaconda3\lib\site-packages\tianshou\env\venvs.py", line 179, in _assert_id
    assert i not in self.waiting_id, \
AssertionError: Cannot interact with environment 7 which is stepping now.

thu-ml / tianshou

AssertionError: Cannot interact with environment 7 which is stepping now. #755