frogatto/Wayland: ASSERT EQ FAILED: glew_status != GLEW_OK: 4 != 0

[smcv]

Prerequisites:

• Debian testing (Debian 12 alpha)
• Video: GNOME 43 in Wayland mode (with Mesa 22.2.0 on AMD Vega, if it matters)
• Audio: Pipewire 0.3.59, with pipewire-pulse emulating PulseAudio
• apt install frogatto (Debian package version 1.3.1+dfsg-6)
• Some relevant libraries:
  • libsdl1.2-compat either 1.2.58-1 (packaged in Debian experimental) or commit 63e4393 (locally built)
  • libsdl2-2.0-0 version 2.24.1+dfsg-1

To reproduce:

• frogatto
• LD_LIBRARY_PATH=.../sdl12-compat/_build frogatto
• SDL_VIDEODRIVER=wayland LD_LIBRARY_PATH=.../sdl12-compat/_build frogatto

Expected result: all work

Actual result: X11 works fine. With sdl12-compat and Wayland:

INFO: sdl12-compat, built on Oct  3 2022 at 09:40:31, talking to SDL2 2.24.1
Frogatto engine version 1.3
LOOKING IN 'modules/frogatto/module.cfg': 1
EXPAND DATA PATHS
SET PREFERENCES PATH: ~/.frogatto/
PARSE ERROR: : Could not find file ./signature.cfg
EXPAND DATA PATHS
Preferences dir: /home/desktop/.frogatto/
get_dir(): /home/desktop/.frogatto/

src/main.cpp:585 ASSERT EQ FAILED: glew_status != GLEW_OK: 4 != 0

Aborted (core dumped)

I think the 4 is probably GLEW_ERROR_NO_GLX_DISPLAY.

[icculus]

GLEW is so frustrating, there's almost never a reason to use it (frogatto looks up a handful of GL extensions with it), and it causes problems like this.

I wonder if there's a value in patching GLEW to use SDL behind the scenes, instead of talking to glX directly, which would free up any games like this to (probably) work with Wayland without even rebuilding them.

Otherwise, the only thing I can do here is add a quirk to force this game to use x11, which is only needed to look up GL extension strings.

[smcv]

I wonder if there's a value in patching GLEW to use SDL behind the scenes, instead of talking to glX directly

In Debian (and Fedora and so on), where open-source games like this one can be recompiled against an updated version of GLEW, perhaps - although GLEW regularly gets a new SONAME (ABI break), and I don't know whether that reflects an API break that would need source code changes.

However, I'm using Debian's old SDL 1.2 games partly to test whether/when we can use sdl12-compat as a replacement for real SDL 1.2 in Debian, but also partly as a stand-in for the unknown number of proprietary games that we can't just recompile. The Steam Runtime provides two GLEW ABIs: 1.6, which is pretty ancient (circa 2012), and 1.10, which is semi-ancient (somewhere around 2014). Games targeting Steam could either be using the 1.6 ABI (which we can patch if we have to), or the 1.10 ABI (which we can patch if we have to), or statically or dynamically linking their own GLEW which we can't patch.

https://github.com/anholt/libepoxy is another alternative to GLEW (notably, GTK uses it).

[smcv]

What if we did something like this during init? That would potentially allow removing one of the quirks for Awesomenauts and Braid as well, which seems like maybe a win?

(This is totally untested so far, I'll look into it further next week if you think this approach has potential)

#ifdef __linux__
void *global_symbols = dlopen(NULL, RTLD_LOCAL|RTLD_NOW);
SDL_bool force_x11 = SDL_FALSE;

/* Use linked libraries to detect what quirks we are likely to need */
if (global_symbols != NULL) {
    /* GLEW (e.g. Frogatto, SLUDGE) */
    force_x11 ||= (dlsym(global_symbols, "glxewInit") != NULL);
    /* NVIDIA Cg (e.g. Awesomenauts, Braid) */
    force_x11 ||= (dlsym(global_symbols, "cgGLEnableProgramProfiles") != NULL);
}

dlclose(global_symbols);

if (force_x11) {
    SDL20_SetHint("SDL_VIDEODRIVER", "x11");
}
#endif

[smcv]

That trick won't work for games that statically link Cg or GLEW (unless they are linked with -Wl,-export-dynamic), and it won't necessarily work for all games that dlopen() things, but it would at least work for games that are linked with -lGLEW or -lCgGL in the normal way - which happens to include Awesomenauts, Braid, Frogatto and sludge-engine.

[slouken]

This seems like a good idea to me. @icculus?

[smcv]

It's a bit unfortunate that this would be forcing X11 even if the game actually does cleverer things to use these libraries in a Wayland-friendly way, but if people want their actively-maintained games to work optimally, then they should be using SDL2 anyway - so it's not necessarily a bad thing to be applying a bit of brute force here.

[icculus]

This is a good idea, I'll play with this a bit.

[sezero]

Do we have any games that play with glX in any way, e.g. glXGetProcAddress or whatever? If so, the check can be move to LoadSDL20Library() to before actually loading SDL2.

BTW, minor nitpick for https://github.com/libsdl-org/sdl12-compat/commit/2a96bb35d7562f75c710e5824a49eb14c13c1bac:

diff --git a/src/SDL12_compat.c b/src/SDL12_compat.c
index cb97408..4b9106e 100644
--- a/src/SDL12_compat.c
+++ b/src/SDL12_compat.c
@@ -1340,7 +1340,7 @@
             /* NVIDIA Cg (e.g. Awesomenauts, Braid) */
             else if (dlsym(global_symbols, "cgGLEnableProgramProfiles") != NULL) { force_x11 = SDL_TRUE; }
-        }

-        dlclose(global_symbols);
+            dlclose(global_symbols);
+        }

         if (force_x11) {

[icculus]

Good idea (and whoops on the formatting!)

[icculus]

Okay, this should be resolved now.

[smcv]

Quirk seems to work, although I haven't verified it with a SDL that has been patched to default to Wayland (like the one in Fedora).