Importing `classic_control.rendering` before making `AsyncVectorEnv` with `Monitor` will cause error

[LiuTed]

I was trying to vectorize my customized environment, which imported gym.envs.classic_control.rendering globally, using gym.vector.make. However, I got errors like the following lines when reseting the envrionment, even using CartPole-v0:

XIO:  fatal IO error 0 (Success) on X server "192.168.208.1:0"
      after 97 requests (97 known processed) with 0 events remaining.
Exception ignored in: <function Viewer.__del__ at 0x7f843aa0d310>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/gym/envs/classic_control/rendering.py", line 185, in __del__
    self.close()
  File "/usr/local/lib/python3.8/dist-packages/gym/envs/classic_control/rendering.py", line 99, in close
    if self.isopen and sys.meta_path:
AttributeError: 'Viewer' object has no attribute 'isopen'
Traceback (most recent call last):
  File "test.py", line 14, in <module>
    env.reset()
  File "/usr/local/lib/python3.8/dist-packages/gym/vector/vector_env.py", line 61, in reset
    return self.reset_wait()
  File "/usr/local/lib/python3.8/dist-packages/gym/vector/async_vector_env.py", line 221, in reset_wait
    results, successes = zip(*[pipe.recv() for pipe in self.parent_pipes])
  File "/usr/local/lib/python3.8/dist-packages/gym/vector/async_vector_env.py", line 221, in <listcomp>
    results, successes = zip(*[pipe.recv() for pipe in self.parent_pipes])
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes
    buf = self._recv(4)
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 383, in _recv
    raise EOFError
EOFError
/usr/local/lib/python3.8/dist-packages/gym/logger.py:34: UserWarning: WARN: Calling `close` while waiting for a pending call to `reset` to complete.

This error can be easily reproduced with the following codes:

import gym
from gym.vector import make
import pyglet.gl # or import gym.envs.classic_control.rendering

env = make('CartPole-v0', 2,
    wrappers=[
        lambda env, i=_i: gym.wrappers.Monitor(
            env,
            'cartpole-log-{}'.format(i),
            force=True
        ) for _i in range(2)]
)

env.reset()

If commanded out the line 3:import pyglet.gl, this error will not happen. This problem will only happens iff using AsyncVectorEnv with num_envs>1, and using Monitor wrapper. If num_envs=1 or asynchronous=False, or simply not using Monitor, this will not happen.

I am wondering why such error will occur, and if it is possible to warn such usage (import rendering globally).

I executed the code above in Python 3.8 with libraries gym==0.19.0, pyglet==1.5.0. OS is Ubuntu 20.04 in WSL2. X Server is VcXsrv 1.20.8.1.

[tristandeleu]

I was not able to reproduce this error, but when you give a list to wrappers in gym.vector.make, this applies the sequence of wrappers to all the sub-environments. Here, this means that for both sub-environments, you apply Monitor twice; in other words, all the sub-environments will be instantiated like

Monitor(Monitor(gym.make('CartPole-v0'), 'cartpole-log-0'), 'cartpole-log-1')

I don't know if this explains your issue though sorry. This worked for me (tested it on MacOS):

import gym
import pyglet.gl # or import gym.envs.classic_control.rendering

def make_env(i):
    def _make():
        env = gym.make('CartPole-v0')
        env = gym.wrappers.Monitor(env, f'cartpole-log-{i}', force=True)
        return env
    return _make

env = gym.vector.AsyncVectorEnv([make_env(i) for i in range(2)])
env.reset()

[LiuTed]

I was not able to reproduce this error, but when you give a list to wrappers in gym.vector.make, this applies the sequence of wrappers to all the sub-environments. Here, this means that for both sub-environments, you apply Monitor twice; in other words, all the sub-environments will be instantiated like

Monitor(Monitor(gym.make('CartPole-v0'), 'cartpole-log-0'), 'cartpole-log-1')

I don't know if this explains your issue though sorry. This worked for me (tested it on MacOS):

import gym
import pyglet.gl # or import gym.envs.classic_control.rendering

def make_env(i):
    def _make():
        env = gym.make('CartPole-v0')
        env = gym.wrappers.Monitor(env, f'cartpole-log-{i}', force=True)
        return env
    return _make

env = gym.vector.AsyncVectorEnv([make_env(i) for i in range(2)])
env.reset()

Thanks for your notice about the vector.make, but this is inrelevent to the problem. I am sorry for not being able to reproduce this problem, maybe it is caused by the implementation of X server. Thanks for your reply.

[tristandeleu]

Alternatively it could be because Linux by default uses the start method fork for multiprocessing, which may not play well with pyglet. You can try to set the start method to something else, e.g. spawn, with

if __name__ == '__main__':
    env = gym.vector.AsyncVectorEnv([make_env(i) for i in range(2)], context='spawn')
    env.reset()

You would have to use the AsyncVectorEnv API for changing the context, this is not accessible from gym.vector.make. Don't forget to wrap your code containing the vectorized environment in if __name__ == '__main__':, see this documentation on multiprocessing for more information.

[LiuTed]

After switching the context to 'spawn', this problem seems to be fixed. Thanks a lot.