Farama-Foundation / Gymnasium

An API standard for single-agent reinforcement learning environments, with popular reference environments and related utilities (formerly Gym)
https://gymnasium.farama.org
MIT License
7.39k stars 836 forks source link

[Bug Report] Async Vec Env raise pickle error for ctypes in MujocoRenderer #1161

Closed Chaoqi-LIU closed 2 months ago

Chaoqi-LIU commented 2 months ago

Describe the bug

as title, when I try to create a async vec env wrapper for metaworld 2.0.0 version, it failed because gymnasium.envs.mujoco.mujoco_rendering involves

def _import_egl(width, height):
    from mujoco.egl import GLContext
    return GLContext(width, height)

in which contains

class GLContext:
  """An EGL context for headless accelerated OpenGL rendering on GPU devices."""

  def __init__(self, max_width, max_height):
    del max_width, max_height  # unused
    num_configs = ctypes.c_long()           <---- ctypes
    config_size = 1
    config = EGL.EGLConfig()
    EGL.eglReleaseThread()
    EGL.eglChooseConfig(
        EGL_DISPLAY,
        EGL_ATTRIBUTES,
        ctypes.byref(config),
        config_size,
        num_configs)
    if num_configs.value < 1:

[skip]

Therefore, we got error from multiprocessing, saying that ValueError: ctypes objects containing pointers cannot be pickled, which then cause EGL / viewers / renderer breaks. When I use SyncVectorEnv as the wrapper, or directly use this without vectorization, everything is fine.

Minimal Code for error reproduction

import metaworld
import gymnasium
from gymnasium.envs.mujoco.mujoco_rendering import MujocoRenderer
from gymnasium.vector.async_vector_env import AsyncVectorEnv

task_name = 'shelf-place-v2-goal-observable'
seed = 42
n_env = 2

class MetaworldEnv(gymnasium.Env):
    metadata = {'render_modes': ['rgb_array', 'depth_array']}

    def __init__(self):
        self.env = metaworld.envs.ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE[task_name](seed=seed)
        self.observation_space = self.env.observation_space
        self.action_space = self.env.action_space
        self.corner_renderer = MujocoRenderer(
            self.env.model, self.env.data, None, 
            128, 128, 1000, None, 'corner', {}
        )
        self.render_mode = 'rgb_array'

    def _get_rgb(self):
        # NOTE: I'm using more than one camera in actual implementation,
        # here is just for error reproduction, so I'm using only one camera
        return {
            'default': self.env.mujoco_renderer.render('rgb_array'),    # <-- This is the default metaworld
            'corner': self.corner_renderer.render('rgb_array'),         #     renderer, and it also cannot
        }                                                               #     work. Mine too.

    def reset(self, **kwargs):
        self.env.reset()
        self.env.reset_model()
        state = self.env.reset()
        obs_dict = self._get_rgb()      # <-- This is the line that causes the error
        obs_dict['full_state'] = state
        return obs_dict

def env_fn():
    return MetaworldEnv()

env = AsyncVectorEnv([env_fn for _ in range(n_env)])
env.reset()

System info

gymnasium 1.0.0a2 metaworld 2.0.0 error can be reproduced on ubuntu 22, macOS Sonoma 14.4.1

Additional context

I intended to have more than one camera, I'm not sure if create more than one renderers (one renderer for one camera) is the correct way to do so, but currently this is causing problem, and I want sensory reading from multiple cameras. Thanks.

Checklist

Chaoqi-LIU commented 2 months ago

Fixed: Farama-Foundation/Metaworld#509