Get raylib to work on device

maxime-desroches commented 1 month ago

Context

We want to switch from QT to raylib for the UI.

On device, we use Wayland with Weston 1.9 (from 2015) as the compositor.

Wayland defines a bunch of protocols to communicate with the compositor. The protocol we currently use is wl_shell. It has been deprecated since at least 2016 and was replaced with xdg_shell. Our version of weston should support xdg_shell as shown when running weston-info on device.

raylib uses GLFW for creating windows/surfaces and handling user inputs. GLFW deprecated wl_shell in version 3.3 in favor of xdg_shell.

(On a side note, raylib disabled wayland support by default last week : https://github.com/raysan5/raylib/pull/4369 . See this issue for a very good overview of wayland support for raylib: https://github.com/raysan5/raylib/issues/4371)

The problem

GLFW does not work on device. Using pre 3.3 version of GLFW result in segfault when calling glfwCreateWindow() with the following backtrace:

#0  0x0000007fb7ce6d78 in wl_proxy_create_wrapper () from /lib/aarch64-linux-gnu/libwayland-client.so.0
#1  0x0000007fb79da8c4 in ?? () from /lib/aarch64-linux-gnu/libEGL_mesa.so.0
#2  0x0000007fb79cd82c in ?? () from /lib/aarch64-linux-gnu/libEGL_mesa.so.0
#3  0x0000007fb79bdcec in ?? () from /lib/aarch64-linux-gnu/libEGL_mesa.so.0
#4  0x0000007fb7f941e0 in _glfwCreateContextEGL () from /data/tmp/glfw/glfw-3.2/build/src/libglfw.so.3
#5  0x0000007fb7f907c4 in _glfwPlatformCreateWindow () from /data/tmp/glfw/glfw-3.2/build/src/libglfw.so.3
#6  0x0000007fb7f8a290 in glfwCreateWindow () from /data/tmp/glfw/glfw-3.2/build/src/libglfw.so.3
#7  0x0000005555555c9c in main ()

It is currently possible to create and open a window (and even do openGL rendering) manually using the wl_shell protocol with https://gist.github.com/Miouyouyou/ca15af1c7f2696f66b0e013058f110b4 and compiling it with gcc -o test init_window.c -I. -lwayland-client -lwayland-server -lwayland-egl -lEGL -lGLESv2 or even simpler without openGL https://gist.github.com/hoyon/e9f86c481528c2eeca22921bad90b60c

Trying to manually create windows using xdg_shell also results in segfault:

#0  0x0000007fb7f99b08 in wl_proxy_marshal_constructor () from /lib/aarch64-linux-gnu/libwayland-client.so.0
#1  0x00000055555572e0 in xdg_wm_base_get_xdg_surface ()
#2  0x0000005555557a84 in main ()

Goal

Make GLFW and raylib work on device

maxime-desroches commented 1 month ago

@deanlee Interested in working on this?

deanlee commented 1 month ago

I'll give it a try. I'll start once the Qt dependency is removed from Replay.

adeebshihadeh commented 3 weeks ago

$300 bounty for getting this PR running on device (performantly) https://github.com/commaai/openpilot/pull/33738

AbowlofS2 commented 3 weeks ago

Hey is this still open I am interested in working on this I come from a gamedev background?

adeebshihadeh commented 3 weeks ago

Yes, it's still open. You can lock it by opening a draft PR with some progress towards it.

commaai / openpilot