VirtualGL and Cinnamon WM crash when window unmaximize

[Tomek-1]

Hello.

This is my envinronment.

System: Debian 11 (Bullseye) GPU: GRID Nvidia RTX8000-8Q (VGPU) - driver ver. 470.82.01 Remote access: XRDP + TigerVNC WM: Cinnamon 4.8.6

Cinnamon session is started via:

vglrun -c proxy -np 2 -d /dev/dri/card1 +sync +wm cinnamon-session-cinnamon

Everything works fine, until you try to unmaximize any window, then the cinnamon crashes. When I replace the cinnamon with gnome-shell then there is no such problem.

Cinnamon Coredump:

Core was generated by `cinnamon --replace'. Program terminated with signal SIGTRAP, Trace/breakpoint trap.

0 0x00007f40d8bfbabb in g_log_writer_default () from /lib/x86_64-linux-gnu/libglib-2.0.so.0

[Current thread is 1 (Thread 0x7f40d2b20c40 (LWP 433159))] (gdb) bt full

0 0x00007f40d8bfbabb in g_log_writer_default () from /lib/x86_64-linux-gnu/libglib-2.0.so.0

No symbol table info available.

1 0x00007f40d8bf9c67 in g_log_structured_array () from /lib/x86_64-linux-gnu/libglib-2.0.so.0

No symbol table info available.

2 0x00007f40d8bfa762 in g_log_structured_standard () from /lib/x86_64-linux-gnu/libglib-2.0.so.0

No symbol table info available.

3 0x00007f40d7dafbca in ?? () from /lib/x86_64-linux-gnu/libgdk-3.so.0

No symbol table info available.

4 0x00007f40d7dbcf53 in ?? () from /lib/x86_64-linux-gnu/libgdk-3.so.0

No symbol table info available.

5 0x00007f40d8947864 in _XError () from /lib/x86_64-linux-gnu/libX11.so.6

No symbol table info available.

6 0x00007f40d8944327 in ?? () from /lib/x86_64-linux-gnu/libX11.so.6

No symbol table info available.

7 0x00007f40d894580c in _XReply () from /lib/x86_64-linux-gnu/libX11.so.6

No symbol table info available.

8 0x00007f40d8366d7f in XShmGetImage () from /lib/x86_64-linux-gnu/libXext.so.6

No symbol table info available.

9 0x00007f40d8017898 in ?? () from /lib/x86_64-linux-gnu/libmuffin-cogl-0.so

No symbol table info available.

10 0x00007f40d80180f4 in ?? () from /lib/x86_64-linux-gnu/libmuffin-cogl-0.so

No symbol table info available.

11 0x00007f40d7fec15d in ?? () from /lib/x86_64-linux-gnu/libmuffin-cogl-0.so

No symbol table info available.

12 0x00007f40d7fda0fb in ?? () from /lib/x86_64-linux-gnu/libmuffin-cogl-0.so

No symbol table info available.

13 0x00007f40d7fe8ca9 in cogl_pipeline_foreach_layer () from /lib/x86_64-linux-gnu/libmuffin-cogl-0.so

No symbol table info available.

14 0x00007f40d7fda9c0 in ?? () from /lib/x86_64-linux-gnu/libmuffin-cogl-0.so

No symbol table info available.

15 0x00007f40d8009411 in cogl_framebuffer_draw_multitextured_rectangle () from /lib/x86_64-linux-gnu/libmuffin-cogl-0.so

No symbol table info available.

16 0x00007f40d882d81d in ?? () from /lib/x86_64-linux-gnu/libmuffin.so.0

No symbol table info available.

17 0x00007f40d882da97 in ?? () from /lib/x86_64-linux-gnu/libmuffin.so.0

No symbol table info available.

18 0x00007f40d8a94b33 in clutter_actor_paint () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

19 0x00007f40d8a94d60 in ?? () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

20 0x00007f40d8a94b33 in clutter_actor_paint () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

21 0x00007f40d8a94d60 in ?? () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

22 0x00007f40d8835032 in ?? () from /lib/x86_64-linux-gnu/libmuffin.so.0

No symbol table info available.

23 0x00007f40d8a94b33 in clutter_actor_paint () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

24 0x00007f40d8f59284 in ?? () from /usr/lib/x86_64-linux-gnu/cinnamon/libcinnamon.so

No symbol table info available.

25 0x00007f40d8a94b33 in clutter_actor_paint () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

26 0x00007f40d8a94d60 in ?? () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

27 0x00007f40d8a94b33 in clutter_actor_paint () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

28 0x00007f40d8ae9c2a in ?? () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

--Type for more, q to quit, c to continue without paging-- No symbol table info available.

29 0x00007f40d8a94b33 in clutter_actor_paint () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

30 0x00007f40d8aea80f in ?? () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

31 0x00007f40d8aea8c3 in ?? () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

32 0x00007f40d8a854ba in ?? () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

33 0x00007f40d8a85f76 in ?? () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

34 0x00007f40d8a86abb in ?? () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

35 0x00007f40d8aeb016 in ?? () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

36 0x00007f40d8ad6459 in ?? () from /usr/lib/x86_64-linux-gnu/muffin/libmuffin-clutter-0.so

No symbol table info available.

37 0x00007f40d8bf2e6b in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0

No symbol table info available.

38 0x00007f40d8bf3118 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0

No symbol table info available.

39 0x00007f40d8bf340b in g_main_loop_run () from /lib/x86_64-linux-gnu/libglib-2.0.so.0

No symbol table info available.

40 0x00007f40d884b8fa in meta_run () from /lib/x86_64-linux-gnu/libmuffin.so.0

No symbol table info available.

41 0x000056087ea377d8 in ?? ()

No symbol table info available.

42 0x00007f40d863ed0a in __libc_start_main (main=0x56087ea37320, argc=2, argv=0x7ffcc3200658, init=, fini=, rtld_fini=,

stack_end=0x7ffcc3200648) at ../csu/libc-start.c:308
    self = <optimized out>
    result = <optimized out>
    unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, -1405534622123388973, 94594484369616, 0, 0, 0, -4640974037182148653, -4688312957596823597}, mask_was_saved = 0}}, priv = {pad = {
          0x0, 0x0, 0x2, 0x7ffcc3200658}, data = {prev = 0x0, cleanup = 0x0, canceltype = 2}}}
    not_first_call = <optimized out>

43 0x000056087ea378fa in ?? ()

[dcommander]

No clue, but I will point out that -np 2 does nothing when using the X11 Transport. Does the crash also occur without +sync and +wm?

[dcommander]

Also, can you clarify what you mean by XRDP + TigerVNC?

[Tomek-1]

Ad. 1 Yes, it also crashes without +sync and +wm.

Ad.2 Windows RDP client -> Linux XRDP Server -> XRDP VNC client -> TigerVNC Server -> XDMCP -> GDM -> Cinnamon Sesson

[dcommander]

That's a lot of moving parts. Can you verify whether or not the crash occurs with the TurboVNC Server and Viewer?

[Tomek-1]

I have tested the following variants:

Windows VNC Client -> TigerVNC Server -> XDMCP -> GDM -> vglrun Cinnamon Sesson Still crashes.

The same but without vglrun works fine. Windows VNC Client -> TigerVNC Server -> XDMCP -> GDM -> Cinnamon Sesson

Other variants with vglrun work fine. Windows VNC Client -> TigerVNC Server -> vglrun Cinnamon Sesson Windows RDP client -> Linux XRDP Server -> XRDP Xorg client -> vglrun Cinnamon Sesson Windows VNC Client -> TigerVNC Server -> XDMCP -> GDM -> vglrun Gnome Shell Sesson

[dcommander]

Given that you are able to make it work without XDMCP, I strongly suspect that it isn't a VGL issue. I develop and support VirtualGL and TurboVNC. I am happy to fix issues between VGL and other X proxies, such as TigerVNC and XRDP. However, I don't have the resources to diagnose issues that are outside of the scope of projects I control, unless an organization is paying for my labor to perform that diagnosis. In other words, this falls under the "Community Support" category (https://virtualgl.org/Documentation/OSSupport). If someone can figure out the problem and submit a PR, I will happily integrate it, but that's probably the limit of what I can do without direct funding. XDMCP isn't well-supported in general these days, so it doesn't surprise me that it has issues.

[Tomek-1]

I'm sorry but I made a mistake in my previous message due to my haste. I wrote that vglrun works in the following configuration:

Windows VNC Client -> TigerVNC Server -> vglrun Cinnamon Sesson

It's not true. Unfortunately, however, it does not work and cinnamon crashes when any window is unmaximized.

So the problem is not related to XMDCP only and also occurs in regular VNC session.

[Tomek-1]

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1036107

[dcommander]

The nearest I could get to reproducing your configuration was Cinnamon 4.4.x under Ubuntu 20.04 running in a VMWare virtual machine with the VMWare Mesa driver, but I could not reproduce the failure with that configuration. That's unfortunately the limit of what I can do at the moment.

[dcommander]

Also unable to reproduce with Ubuntu 22.04 and Cinnamon 5.2.x.

[dcommander]

I cannot reproduce the issue on your system using TurboVNC 3.0.3. That means that either I have misunderstood how the issue can be reproduced or the issue is specific to TigerVNC or xrdp and is thus out of scope for my support.

[Tomek-1]

Ok. I understand. Thank you for your support.

Regards. Tom

[dcommander]

I am happy to log in again and retest if you can narrow down the scope of the issue. I wasn't completely clear as to what you meant by "unmaximize." I maximized and then restored several windows, including a terminal and file manager window, and didn't observe the issue.

[dcommander]

I definitely do see a crash with xrdp, but I did not see the crash with TurboVNC while performing the same operation. That may mean that there is a bug in the X.org code base used by your particular version of Debian. (TigerVNC and xrdp are built using the system's X.org code base, whereas TurboVNC has its own X.org code base.)

[Tomek-1]

Thanks for the explanation.

In the near future, I will be testing the same configuration in the newer Debian 12. Let's hope that the described problem will not occur there

[dcommander]

For now, I am closing the issue as SEP. If you are ever able to reproduce the problem using TurboVNC, then I can reconsider the issue.

[Tomek-1]

OK.

BTW. Can you remove my public IP address from github email dialog?

Regards.

[dcommander]

OK. BTW. Can you remove my public IP address from github email dialog? Regards.

Done