[SDL2] testautomation video_createWindowVariousFlags failing under dummy driver on Linux

To reproduce:

Debian 12 in a qemu/kvm virtual machine (running GNOME 43 in Wayland mode, if that matters)
SDL 3 libsdl-org/SDL@4db2b968 in ~/src/SDL3, configured with cmake -S . -B _build -DSDL_TESTS=ON and compiled
SDL 2 libsdl-org/SDL@9772d051 in ~/src/SDL2, configured with cmake -S . -B _build -DSDL_TESTS=ON and compiled
sdl2-compat libsdl-org/sdl2-compat@52d96367 in ~/src/sdl2-compat, configured with cmake -S . -B _build -DSDL3_INCLUDE_DIRS=$HOME/src/SDL3/include -DSDL2COMPAT_TESTS=ON and compiled
export SDL_VIDEODRIVER=dummy SDL_AUDIODRIVER=dummy SDL_VIDEO_DRIVER=dummy SDL_AUDIO_DRIVER=dummy
cd ~/src/SDL3/_build/test; ./testautomation (all succeeds)
cd ~/src/sdl2-compat/_build; LD_LIBRARY_PATH=$HOME/src/SDL3/_build ./testautomation
cd ~/src/SDL2/_build/test; ./testautomation

Expected result: SDL2 test suite passes

Actual result: both SDL2 and sdl2-compat have several test failures, most of which are out of scope for this particular bug report. The one I'm interested in right now is video_createWindowVariousFlags, in which creating a window with flag 2 (SDL_WINDOW_OPENGL) is failing:

INFO:  09/28/23 11:26:25: ----- Test Case 21.4: 'video_createWindowVariousFlags' started
INFO:  09/28/23 11:26:25: Test Description: 'Create windows using various flags'
INFO:  09/28/23 11:26:25: Test Iteration 1: execKey 153473147769168775
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_CreateWindow('Title',82,57,985,651,2)': Passed
ERROR: 09/28/23 11:26:25: Assert 'Validate that returned window struct is not NULL': Failed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_CreateWindow('Title',82,57,985,651,4)': Passed
INFO:  09/28/23 11:26:25: Assert 'Validate that returned window struct is not NULL': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_DestroyWindow()': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_CreateWindow('Title',82,57,985,651,8)': Passed
INFO:  09/28/23 11:26:25: Assert 'Validate that returned window struct is not NULL': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_DestroyWindow()': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_CreateWindow('Title',82,57,985,651,16)': Passed
INFO:  09/28/23 11:26:25: Assert 'Validate that returned window struct is not NULL': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_DestroyWindow()': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_CreateWindow('Title',82,57,985,651,32)': Passed
INFO:  09/28/23 11:26:25: Assert 'Validate that returned window struct is not NULL': Passed 
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_DestroyWindow()': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_CreateWindow('Title',82,57,985,651,64)': Passed
INFO:  09/28/23 11:26:25: Assert 'Validate that returned window struct is not NULL': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_DestroyWindow()': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_CreateWindow('Title',82,57,985,651,128)': Passed
INFO:  09/28/23 11:26:25: Assert 'Validate that returned window struct is not NULL': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_DestroyWindow()': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_CreateWindow('Title',82,57,985,651,256)': Passed
INFO:  09/28/23 11:26:25: Assert 'Validate that returned window struct is not NULL': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_DestroyWindow()': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_CreateWindow('Title',82,57,985,651,512)': Passed
INFO:  09/28/23 11:26:25: Assert 'Validate that returned window struct is not NULL': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_DestroyWindow()': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_CreateWindow('Title',82,57,985,651,1024)': Passed
INFO:  09/28/23 11:26:25: Assert 'Validate that returned window struct is not NULL': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_DestroyWindow()': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_CreateWindow('Title',82,57,985,651,2048)': Passed
INFO:  09/28/23 11:26:25: Assert 'Validate that returned window struct is not NULL': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_DestroyWindow()': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_CreateWindow('Title',82,57,985,651,1048576)': Passed
INFO:  09/28/23 11:26:25: Assert 'Validate that returned window struct is not NULL': Passed
INFO:  09/28/23 11:26:25: Assert 'Call to SDL_DestroyWindow()': Passed
INFO:  09/28/23 11:26:25: Fuzzer invocations: 4
ERROR: 09/28/23 11:26:25: Assert Summary: Total=35 Passed=34 Failed=1
INFO:  09/28/23 11:26:25: Total Test runtime: 0.0 sec
ERROR: 09/28/23 11:26:25: >>> Test 'video_createWindowVariousFlags': Failed

The same is true for sdl2-compat, so I suppose it's faithfully emulating real SDL2, which is sort of good... but still a test failure.

Do we expect SDL_WINDOW_OPENGL with the dummy driver to fail, in which case that particular test should be skipped when using the dummy driver? Or do we expect it to succeed? (My test VM does not have a real GPU, but it does have GLX/EGL rendering on the CPU via Mesa's llvmpipe, which works well enough for GNOME Shell and glxgears.)

I'm reporting this because eventually, I'd like to be able to use testautomation as an acceptance test for new SDL versions in Debian. It looks as though I can now do this for SDL3 (at least with the dummy drivers), but not with SDL2 yet.

libsdl-org / SDL

[SDL2] testautomation video_createWindowVariousFlags failing under dummy driver on Linux #8309