Capturing SDL offscreen mode skips roughly 9/10 frames on HD Graphics

[scaledteam]

Recently i tried obs-vkcapture with offscreen EGL. It works really well in some programs, for example here: https://github.com/peko/egl-headless-render

But it works very poorly for SDL applications. To test it, compile SDL 2.26.x with cmake .. -DSDL_TESTS=ON . Latest works the same, but environment varriables is different, so just in case 2.26.x.

Then you can run OpenGL2 test from SDL with this command:

SDL_VIDEODRIVER=offscreen SDL_HINT_EGL_DEVICE=2 strangle 30 obs-gamecapture test/testgl2

Some notes:

• You need to use Strangle or other frame limiting software to properly test it, othervise you will overload your GPU and results will not be very representative.
• When set really high framerate limiter (600 for example), it looks normal in OBS.
• On multi-gpu system you need to set SDL_HINT_EGL_DEVICE=2 to select HD Graphics, othervise Nvidia GPU will be selected
• Program seems to generate all frames, but only 9/10 of them captured in OBS.
• When run with SDL_HINT_EGL_DEVICE=0, it defaults to Nvidia GPU, which works perfect when OBS is also run in Nvidia GPU (520.56.06 driver). Maybe it's related to this: https://github.com/nowrep/obs-vkcapture/issues/70 , but in my case it able to capture 60 frames per second absolutely fine.

By the way, same programs works fine on HD Graphics when run in usual mode, when rendered content displayed on screen. I interested in offscreen rendering because i want to hide my VTuber software from desktop and render it only to OBS, without rendering on screen. Maybe it will require patch to SDL or to my software, based on it, but i will be glad to hear any suggestions.

[nowrep]

It's because there is no explicit glFlush call in the demo, and offscreen context seem to not do implicit flush on swap buffers.

What's the point of offscreen swapchain anyway? Normally you would just render to texture and then maybe read the texture data, but with offscreen swapchain you're just throwing the output away?

[scaledteam]

Thanks for responding. My VTuber software based on Urho3D. I did many attempts to implement rendering in texture with this engine, but i never succeeded. And recently i finally found that i can use SDL offscreen rendering, render output somewhere in the video memory, and your plugin will find it and display in OBS. Not as efficient as rendering directly to OBS, but at least it bypasses window manager and probably save some bandwidth. What is glFlush? Can it be added somewhere in SDL to make capture more reliable? And also, why Nvidia driver is not affected?

[nowrep]

You can just use this patch:

diff --git a/src/glinject.c b/src/glinject.c
index d4339ac..93ef5bf 100644
--- a/src/glinject.c
+++ b/src/glinject.c
@@ -251,6 +251,8 @@ static void gl_copy_backbuffer(GLuint dst)

 static void gl_shtex_capture()
 {
+    glFlush();
+
     GLboolean last_srgb;
     GLint last_read_fbo;
     GLint last_draw_fbo;

And also, why Nvidia driver is not affected?

https://registry.khronos.org/EGL/sdk/docs/man/html/eglSwapBuffers.xhtml

eglSwapBuffers performs an implicit flush operation on the context (glFlush for an OpenGL ES or OpenGL context, vgFlush for an OpenVG context) bound to surface before swapping

Possibly this is a mesa bug? I'm not really sure if this also applies to offscreen context.

[scaledteam]

Thank you so much!!! It works!!! I knew you the right person to ask about it.

It's because there is no explicit glFlush call in the demo

Yesterday i watched code of the egl-headless-render and SDL and i not found any glFlush() near eglSwapBuffers function. But for some reason, egl-headless-render works fine without it, but SDL is not.

Maybe i will add this function explicitly for my patched version of SDL (Urho3D uses very old SDL, i had to port offscreen patch to it). But for now i will leave glFlush in your plugin. Thank you for help!

[nowrep]

Yesterday i watched code of the egl-headless-render and SDL and i not found any glFlush() near eglSwapBuffers function. But for some reason, egl-headless-render works fine without it, but SDL is not.

https://github.com/peko/egl-headless-render/blob/b8ffabd6f4496c1c06a68029255f186afdde7709/scene.c#L129