haoyu-x commented 3 years ago

Hi, thanks for sharing this amazing project!

I'm trying to use env.render(mode = 'rgb_array') to return images on a server. File "/h/haoyux/metaworld/metaworld/envs/mujoco/mujoco_env.py", line 131, in _get_viewer self.viewer = mujoco_py.MjViewer(self.sim) File "/scratch/gobi1/haoyux/gaifo_ae/lib/python3.6/site-packages/mujoco_py/mjviewer.py", line 137, in init super().init(sim) File "/scratch/gobi1/haoyux/gaifo_ae/lib/python3.6/site-packages/mujoco_py/mjviewer.py", line 28, in init super().init(sim) File "mujoco_py/mjrendercontext.pyx", line 311, in mujoco_py.cymj.MjRenderContextWindow.init File "mujoco_py/mjrendercontext.pyx", line 46, in mujoco_py.cymj.MjRenderContext.init File "mujoco_py/mjrendercontext.pyx", line 102, in mujoco_py.cymj.MjRenderContext._setup_opengl_context File "mujoco_py/opengl_context.pyx", line 44, in mujoco_py.cymj.GlfwContext.init File "mujoco_py/opengl_context.pyx", line 64, in mujoco_py.cymj.GlfwContext._init_glfw mujoco_py.cymj.GlfwError: Failed to initialize GLFW

I'm wondering how to do off-screen render. (e.g., robosuite has an option of offscreen render, so you don't need to open a glfw window to call render)

Thanks!

ryanjulian commented 3 years ago

@haydenshively may be able to help with this.

It looks like the robosuite implementation dynamically replaces the render with an off-screen instance here: https://github.com/ARISE-Initiative/robosuite/blob/master/robosuite/environments/base.py#L222

Note that I think you can render images using the default render if you run metaworld using a fake display server, e.g. Xvfb.

You can see an example here in the garage Dockerfile: https://github.com/rlworkgroup/garage/blob/master/docker/entrypoint-headless.sh#L18

haydenshively commented 3 years ago

I'm not 100% sure whether this will work on a headless machine (though I suspect it will), but we use an offscreen rendering mode in scripts/scripted_policy_movies.ipynb -- the line looks like this:

yield r, done, info, env.sim.render(*res, mode='offscreen', camera_name=camera)

I think you should be able to adapt this to your needs. If not, let us know and we'll be happy to help more

haoyu-x commented 3 years ago

thanks for your reply! I solved this by:

[ def _get_viewer(self, mode): self.viewer = self._viewers.get(mode) if self.viewer is None: if mode == 'human': self.viewer = mujoco_py.MjViewer(self.sim) elif 'rgb_array' in mode: self.viewer = mujoco_py.MjRenderContextOffscreen(self.sim, device_id=-1) self.viewer_setup() self._viewers[mode] = self.viewer

if mode == 'rgb_array_y':

    #     self.viewer_setup(view_angle='y')
    # else:
    #     self.viewer_setup(view_angle='x')
    #self.viewer_setup()
    return self.viewer](url)

in /metaworld/metaworld/mujoco_env.py

Farama-Foundation / Metaworld

off screen render #273

if mode == 'rgb_array_y':