OpenGL rendering issue on a headless setup

[Coder-RG]

Hello!

I am trying to run simulations on a headless Raspberry Pi 4 to collect some training data for training a control policy for a quadrotor. The intended objective of the script is to collect necessary observation data from the environment and also the renders. The python script runs fine on my mac but shows the following error on the RPi.

/home/pi/Quadrotor/.venv/lib/python3.9/site-packages/glfw/__init__.py:912: GLFWError: (65544) b'X11: The DISPLAY environment variable is missing'
  warnings.warn(message, GLFWError)
/home/pi/Quadrotor/.venv/lib/python3.9/site-packages/glfw/__init__.py:912: GLFWError: (65537) b'The GLFW library is not initialized'
  warnings.warn(message, GLFWError)
Traceback (most recent call last):
  File "/home/pi/Quadrotor/./rltool", line 237, in <module>
    import generate_dataset as gd
  File "/home/pi/Quadrotor/generate_dataset.py", line 71, in <module>
    renderer = Renderer(env, height=1080, width=1920)
  File "/home/pi/Quadrotor/.venv/lib/python3.9/site-packages/mujoco/renderer.py", line 86, in __init__
    self._mjr_context = _render.MjrContext(
mujoco.FatalError: an OpenGL platform library has not been loaded into this process, this most likely means that a valid OpenGL context has not been created before mjr_makeContext was called

The error mentions something about the DISPLAY variable missing and I even tried to search online for some solution but nothing worked. Also, I did create an OpenGL context, at the beginning of the script, as follows:

import mujoco
# other imports
...
ctx = GLContext(1920, 1080)
ctx.make_current()
# my code
...

Setup details

Hardware: Raspberry Pi 4 (2GB RAM) Mujoco: 2.3.1.post1 Python: 3.9.2

Gist containing both the XML file and the python script -> gist

[DavidPL1]

If you are running a headless (i.e. server) distro, it has probably no X server installed. OpenGL, which is used by mujoco for rendering, requires a running X server and a dispaly. On a headless machine you can use Xvfb to setup a virtual display. Running the following commands before starting your script should do the trick

# Creates a vitual display at :1 and detaches the process to let it run in the background
Xvfb :1 &
# Makes sure following commands in this terminal will use the virtual display 
export DISPLAY=:1

In case you are launching this in a bash script you can additionally install x11-utils and add the following snippet at the very top of your script, to only run your python script once the display is initialized or timeout after 10 seconds

Xvfb :1 &

display_ready=0
tries=0

while [ $display_ready = 1 ] || [ $tries -lt 10 ]
do
    if xdpyinfo -display :1 >& /dev/null ; then break
    else echo "Virutal display not ready yet (try $tries)" && sleep 1 ; fi
    tries=`expr $tries + 1`
done

if xdpyinfo -display :1 >& /dev/null ; then echo "Display exists"
else echo "Display invalid, too many tries" && exit -1 ; fi

export DISPLAY=:1

[Coder-RG]

This works! @DavidPL1 Thank you so much for your help.