[Feature Request] Add Pixel-level Observations for mujoco

[Benjamin-eecs]

Motivation

https://github.com/sail-sg/envpool/pull/107#issuecomment-1123914969

Solution

Init thought:

Add method render or off_screen_render in both mujoco_env.h

• [ ] dmc OffScreenRender
• [ ] gym OffScreenRender
• [ ] FrameStack https://github.com/sail-sg/envpool/issues/148#issuecomment-1143788374

Additional context

mujoco_py use MjRenderPool for multiprocess off-screen and on-screen render on multi-GPU

Road Map

• [x] Go through their record.cc, get familar with off screen render with 3 different OpenGL context: GLFW, OSMesa and EGL
• [x] Update BUILD file
• [x] dm_control: Physics.render and Camera
• [ ] mujoco_py
• [ ] Add some unit tests (good to submit the first PR here);

Resource

dm_control

• Physics.render https://github.com/deepmind/dm_control/blob/main/dm_control/mujoco/engine.py
• Camera https://github.com/deepmind/dm_control/blob/main/dm_control/mujoco/engine.py

mujoco_py

• related issue https://github.com/openai/mujoco-py/issues/201
• mujoco_py.MjRenderContextOffscreen https://github.com/openai/mujoco-py/blob/master/mujoco_py/mjrendercontext.pyx
• off-screen rendering acceleration https://github.com/openai/mujoco-py/blob/master/mujoco_py/mjrenderpool.py

FrameStack Wrapper

• FrameStack

  Checklist

• [x] I have checked that there is no similar issue in the repo (required)

[zhongwen]

An internal project by https://github.com/Ericonaldo requires this feature. Can we bump the priority up? Thanks.