Qt5 + XCB insufficient faking?

[tpwrules]

I am running into an issue where Qt5 apps won't start due to the xcb functions claiming GL is not available. I've managed to fix it with a patch (detailed below) but I'm hoping for a hint on why the issue occurs and if VirtualGL needs a fix or my system is broken.

The software setup is rather byzantine, but here is some relevant info:

• NVIDIA x86_64 545.23.08
• VirtualGL 3.1.1 (using the EGL backend, my system doesn't support the 3D X server)
• TurboVNC 3.1.1
• twm 1.0.12
• Linux 4.18.0-477.10.1.el8_8.x86_64

The Qt GLX backend causes apps to think GL is not available as the XCB functions reply that it isn't, and the Qt EGL backend causes them to render with a black screen. If I patch the XCB functions as below (and enable tracing), then the Qt XCB backend works great:

diff --git a/server/faker-xcb.cpp b/server/faker-xcb.cpp
index 6d875604..58650111 100644
--- a/server/faker-xcb.cpp
+++ b/server/faker-xcb.cpp
@@ -330,6 +330,7 @@ const xcb_query_extension_reply_t *
        STOPTRACE();
        if(reply)
        {
+           ((xcb_query_extension_reply_t*)reply)->present = 1;
            PRARGI(reply->present);  PRARGI(reply->major_opcode);
            PRARGI(reply->first_event);  PRARGI(reply->first_error);
        }
@@ -405,8 +406,14 @@ xcb_glx_query_version_reply_t *
        if(*error) PRARGERR(*error)  else PRARGX(*error);
    }
    else PRARGX(error);
+   if (!reply)
+   {
+       reply = (xcb_glx_query_version_reply_t*)malloc(sizeof(xcb_glx_query_version_reply_t));
+   }
    if(reply)
    {
+       reply->major_version = 4;
+       reply->minor_version = 6;
        PRARGI(reply->major_version);  PRARGI(reply->minor_version);
    }
    else PRARGX(reply);

Any idea why this might be necessary? Is it possible to load the wrong version of the xcb libraries or something?

[dcommander]

With the Qt5 OpenGL examples, I can reproduce the black screen when QT_XCB_GL_INTEGRATION=xcb_egl, regardless of the VirtualGL back end used. (I will look into that.) When QT_XCB_GL_INTEGRATION=xcb_glx, I don't see any visible issues, but xcb_glx_query_version_reply() does return NULL when using VirtualGL's EGL back end. I'm not sure why that is the case, but VirtualGL's EGL back end somewhat hackishly uses the 2D X server's GLX extension to satisfy XCB requests.

Are you doing anything "special" with TurboVNC, such as disabling its GLX extension?

Can you provide an example that demonstrates the issue, or is it reproducible with any of the stock Qt5 examples?

Also, based on my testing, I understand the modifications you made above to xcb_glx_query_version_reply() but not the modifications you made to xcb_get_extension_data(). For me, even when xcb_glx_query_version_reply() returns NULL, reply->present is 1 in xcb_get_extension_data().

Also, xcb_glx_query_version*() is querying the GLX version, not the OpenGL version. It should return 1.4, not 4.6.

fakerut actually tests for exactly this issue, so I'm not sure why Qt5 behaves differently.

I think that, ideally, VirtualGL's XCB interposer shouldn't rely on the 2D X server at all, but it would still be nice to reproduce exactly what you're seeing so that I can understand the scope of the issue.

[dcommander]

Blerg. Scratch that. xcb_glx_query_version_reply() is not actually returning NULL in my testing. I was just derping on processing the trace output. So I'm back to needing a reproducer.

[tpwrules]

I had issues with several Qt5 apps. One specific program I encountered trouble with was CloudCompare. Without the patch it gives an error dialog that OpenGL is not available and it cannot start. With the patch it works fine. (It also reproduces the black screen issue).

By the stock Qt5 examples do you mean these here? I will see if they work properly on my system but I would guess not. I turned on the Qt logging with the environment variable QT_LOGGING_RULES=qt.qpa.gl.debug=true and the failures are directly in the XCB extension; I am not sure how the app uses it matters much.

Good to know about the GLX version, I was just guessing at that point. I don't think I'm using a 2D X server directly (I guess TurboVNC functions as one) and will check if something funky is going on there.

[dcommander]

By the stock Qt5 examples do you mean these here? I will see if they work properly on my system but I would guess not. I turned on the Qt logging with the environment variable QT_LOGGING_RULES=qt.qpa.gl.debug=true and the failures are directly in the XCB extension; I am not sure how the app uses it matters much.

Yes, those are the ones. On Rocky Linux 8, which is what I'm using for testing, there is a prebuilt package with them.

Good to know about the GLX version, I was just guessing at that point. I don't think I'm using a 2D X server directly (I guess TurboVNC functions as one) and will check if something funky is going on there.

Yes, TurboVNC is the 2D X server. That terminology is from the VirtualGL User's Guide.

[dcommander]

I built CloudCompare from source, and it works fine for me with Qt's XCB/GLX back end and VirtualGL's EGL back end. (I even tried it with TWM.) The only other major differences I see are that I'm running EL 8.10 instead of 8.8 and nVidia 550.xx instead of 545.xx, but I don't know why that would matter.

[dcommander]

The blank window with QT_XCB_GL_INTEGRATION=xcb_egl occurs on the local display without VGL when using an nVidia GPU. However, it doesn't occur (either on the local display without VGL or with TurboVNC+VGL) when using an AMD GPU. Thus, methinks that that is an nVidia issue, not a VirtualGL issue.