Flickering/Frozen windows with offscreen decoding

[ToppDev]

Summary

Since updating xpra from 4.3.4-r0-1 to 4.4.3-r0-1 and xpra-html5 from 5.0-r1276-1 to 6.1-r5-1 windows flicker in all browsers except Chrome. Additionally, windows freeze in all browsers after some time (also in Chrome). See screenshots below.

• When using the xpra client to connect, there is not flickering/freezes. So it is related to the browser.
• When disabling offscreen decoding the flickering is gone and also no freezes.

Steps to reproduce

1. Connect to the xpra server in a browser e.g. Firefox
2. Start any application there, e.g. a terminal
3. Click or type in the window to make it flicker

Screenshots

Flickering window on user input (in Firefox)


Frozen window (in Chrome)


Relevant logs

xpra showconfig (only non-default values)

``` bandwidth-limit (used) = '0' bandwidth-limit (default) = 'auto' compressors (used) = 'lz4', 'zlib', 'brotli' compressors (default) = 'none', 'lz4', 'zlib', 'brotli' exit-with-client (used) = True exit-with-client (default) = False file-transfer (used) = 'no' file-transfer (default) = 'auto' html (used) = 'no' html (default) = 'auto' mdns (used) = False mdns (default) = True open-files (used) = 'no' open-files (default) = 'auto' open-url (used) = 'no' open-url (default) = 'auto' pdf-printer (used) = '' pdf-printer (default) = '/usr/share/ppd/cupsfilters/Generic-PDF_Printer-PDF.ppd' postscript-printer (used) = '' postscript-printer (default) = 'drv:///sample.drv/generic.ppd' printing (used) = 'no' printing (default) = 'yes' socket-dirs (used) = '$XDG_RUNTIME_DIR/xpra', '/run/xpra', '~/.xpra' socket-dirs (default) = '/run/user/0/xpra', '/run/xpra', '~/.xpra' sync-xvfb (used) = False sync-xvfb (default) = webcam (used) = 'no' webcam (default) = 'auto' xvfb (used) = 'Xvfb +extension GLX +extension Composite -screen 0 8192x4096x24+32 -nolisten tcp -noreset -auth $XAUTHORITY -dpi 96' xvfb (default) = 'Xvfb +extension GLX +extension Composite -screen 0 8192x4096x24+32 -nolisten tcp -noreset -auth $XAUTHORITY -dpi 96' ```

/etc/defaults/xpra

``` BIND_TCP=0.0.0.0:10000 #BIND_TCP=0.0.0.0:80 TCP_AUTH=sqlite:filename=/etc/xpra/xpra-auth.sdb BIND=none AUTH=none HTML=on #DEBUG=auth,proxy,util,x11 #DEBUG=auth,proxy,x11 DEBUG=auth #skip datetime prefix since the log goes to a system logger: XPRA_LOG_FORMAT="%(message)s" #permissions and group ownership of /run/xpra XPRA_SOCKET_DIR_MODE=775 XPRA_SOCKET_DIR_GROUP=xpra ```

/usr/lib/systemd/system/xpra.service

``` [Unit] Description=Xpra System Server Wants=avahi-daemon.socket Documentation=https://github.com/Xpra-org/xpra/blob/master/docs/Usage/Service.md man:xpra After=network.target xpra.socket Requires=xpra.socket [Service] Type=simple EnvironmentFile=-/etc/default/xpra ExecStart=/usr/bin/xpra proxy :14500 --daemon=no \ --bind-tcp=${BIND_TCP} --tcp-auth=${TCP_AUTH} \ --socket-dirs=/run/xpra --socket-permissions=666 \ --log-dir=/var/log --pidfile=/run/xpra/proxy.pid --debug=${DEBUG} \ --html=${HTML} SuccessExitStatus=0 143 Restart=on-abnormal PIDFile=/run/xpra/proxy.pid ProtectSystem=strict ReadWritePaths=/run/xpra /tmp #PrivateDevices=true ProtectKernelTunables=true ProtectControlGroups=true [Install] WantedBy=multi-user.target ```

Environment info

• OS: Debian GNU/Linux 11 (bullseye) 5.15.74-1 x86_64
• xpra: 4.4.3-r0-1
• xpra-html5: 6.1-r5-1
• Firefox: Mozilla Firefox 107.0.1
• Chrome: Google Chrome 108.0.5359.94

[totaam]

@TijZwa have you seen this?

[TijZwa]

@ToppDev Can you also post a screenshot of the developer console in Chrome? Does it report any errors?

[TijZwa]

Misread the issue; it happens when using browsers other then Chrome. I can reproduce.

[ToppDev]

Yes it is 2 things happening

• flickering in all browsers except Chrome
• unresponsive/freezing windows in all browsers

but both things are solved by disabling offscreen decoding

Chrome Console log

Here a video and chrome console log for the freezing behavior.

https://user-images.githubusercontent.com/17766093/207822651-2ef774c1-89ff-4b91-87a9-7d9859a96e54.mp4

Console output

- Open Thunar window ```javascript OffscreenDecodeWorker.js:246 canvas transfer for window 18 : OffscreenCanvas {width: 1042, height: 482, oncontextlost: null, oncontextrestored: null} false Utilities.js:1 XpraClient {container: div#screen, protocol: XpraProtocolWorkerHost, host: '', port: '443', ssl: true, …} 18 32 419 1042 482 {has-alpha: true, title: 'mitarbeiter', class-instance: Array(2), window-type: Array(1), size-constraints: {…}, …} false false {} (window) => this.send_configure_window(window, {}, false) (event, window) => this.on_mousemove(event, window) (event, window) => this.on_mousedown(event, window) (event, window) => this.on_mouseup(event, window) (event, window) => this.on_mousescroll(event, window) (window) => this._window_set_focus(window) (window) => this.send([PACKET_TYPES.close_window, window.wid]) 1 PaintWorker.js:188 canvas transfer for window 18 : OffscreenCanvas {width: 1042, height: 482, oncontextlost: null, oncontextrestored: null} false ``` - Go with mouse over the items in the right list ```javascript Utilities.js:1 decode error on throttle packet sequence 42 : TypeError: Failed to execute 'postMessage' on 'Worker': Failed to convert value to 'object'. Utilities.js:1 decode error packet: (7) ['damage-sequence', 42, 18, 171, 121, -1, "TypeError: Failed to execute 'postMessage' on 'Worker': Failed to convert value to 'object'."] ``` - Click on the items in the right list (no additional log) - Move mouse to the left list, to the item `Desktop`. A tooltip is appearing. ```javascript OffscreenDecodeWorker.js:246 canvas transfer for window 19 : OffscreenCanvas {width: 185, height: 45, oncontextlost: null, oncontextrestored: null} false PaintWorker.js:188 canvas transfer for window 19 : OffscreenCanvas {width: 185, height: 45, oncontextlost: null, oncontextrestored: null} false Utilities.js:1 XpraClient {container: div#screen, protocol: XpraProtocolWorkerHost, host: '', port: '443', ssl: true, …} 19 21 602 185 45 {title: 'Thunar', class-instance: Array(2), transient-for-xid: 10485767, transient-for: 18, window-type: Array(1), …} true false {} (window) => this.send_configure_window(window, {}, false) (event, window) => this.on_mousemove(event, window) (event, window) => this.on_mousedown(event, window) (event, window) => this.on_mouseup(event, window) (event, window) => this.on_mousescroll(event, window) (window) => this._window_set_focus(window) (window) => this.send([PACKET_TYPES.close_window, window.wid]) 1 ``` - Click on the item `Desktop` ```javascript Utilities.js:1 lost window, was tray= false Utilities.js:1 lost window 19 , remaining: ['18'] Utilities.js:1 XpraClient {container: div#screen, protocol: XpraProtocolWorkerHost, host: '', port: '443', ssl: true, …} 18 32 419 1042 482 {has-alpha: true, title: 'mitarbeiter', class-instance: Array(2), window-type: Array(1), size-constraints: {…}, …} false false {} (window) => this.send_configure_window(window, {}, false) (event, window) => this.on_mousemove(event, window) (event, window) => this.on_mousedown(event, window) (event, window) => this.on_mouseup(event, window) (event, window) => this.on_mousescroll(event, window) (window) => this._window_set_focus(window) (window) => this.send([PACKET_TYPES.close_window, window.wid]) 1 OffscreenDecodeWorker.js:246 canvas transfer for window 20 : OffscreenCanvas {width: 185, height: 45, oncontextlost: null, oncontextrestored: null} false Utilities.js:1 XpraClient {container: div#screen, protocol: XpraProtocolWorkerHost, host: '', port: '443', ssl: true, …} 20 16 593 185 45 {title: 'Thunar', class-instance: Array(2), transient-for-xid: 10485767, transient-for: 18, window-type: Array(1), …} true false {} (window) => this.send_configure_window(window, {}, false) (event, window) => this.on_mousemove(event, window) (event, window) => this.on_mousedown(event, window) (event, window) => this.on_mouseup(event, window) (event, window) => this.on_mousescroll(event, window) (window) => this._window_set_focus(window) (window) => this.send([PACKET_TYPES.close_window, window.wid]) 1 PaintWorker.js:188 canvas transfer for window 20 : OffscreenCanvas {width: 185, height: 45, oncontextlost: null, oncontextrestored: null} false ``` - Move mouse to the item `Trash`. A tooltip is appearing. ```javascript Utilities.js:1 lost window, was tray= false Utilities.js:1 lost window 20 , remaining: ['18'] OffscreenDecodeWorker.js:246 canvas transfer for window 21 : OffscreenCanvas {width: 123, height: 45, oncontextlost: null, oncontextrestored: null} false PaintWorker.js:188 canvas transfer for window 21 : OffscreenCanvas {width: 123, height: 45, oncontextlost: null, oncontextrestored: null} false Utilities.js:1 XpraClient {container: div#screen, protocol: XpraProtocolWorkerHost, host: '', port: '443', ssl: true, …} 21 13 613 123 45 {title: 'Thunar', class-instance: Array(2), transient-for-xid: 10485767, transient-for: 18, window-type: Array(1), …} true false {} (window) => this.send_configure_window(window, {}, false) (event, window) => this.on_mousemove(event, window) (event, window) => this.on_mousedown(event, window) (event, window) => this.on_mouseup(event, window) (event, window) => this.on_mousescroll(event, window) (window) => this._window_set_focus(window) (window) => this.send([PACKET_TYPES.close_window, window.wid]) 1 ``` - Click on the item `Trash` ```javascript Utilities.js:1 lost window, was tray= false Utilities.js:1 lost window 21 , remaining: ['18'] Utilities.js:1 XpraClient {container: div#screen, protocol: XpraProtocolWorkerHost, host: '', port: '443', ssl: true, …} 18 32 419 1042 482 {has-alpha: true, title: 'mitarbeiter', class-instance: Array(2), window-type: Array(1), size-constraints: {…}, …} false false {} (window) => this.send_configure_window(window, {}, false) (event, window) => this.on_mousemove(event, window) (event, window) => this.on_mousedown(event, window) (event, window) => this.on_mouseup(event, window) (event, window) => this.on_mousescroll(event, window) (window) => this._window_set_focus(window) (window) => this.send([PACKET_TYPES.close_window, window.wid]) 1 OffscreenDecodeWorker.js:246 canvas transfer for window 22 : OffscreenCanvas {width: 123, height: 45, oncontextlost: null, oncontextrestored: null} false Utilities.js:1 XpraClient {container: div#screen, protocol: XpraProtocolWorkerHost, host: '', port: '443', ssl: true, …} 22 45 620 123 45 {title: 'Thunar', class-instance: Array(2), transient-for-xid: 10485767, transient-for: 18, window-type: Array(1), …} true false {} (window) => this.send_configure_window(window, {}, false) (event, window) => this.on_mousemove(event, window) (event, window) => this.on_mousedown(event, window) (event, window) => this.on_mouseup(event, window) (event, window) => this.on_mousescroll(event, window) (window) => this._window_set_focus(window) (window) => this.send([PACKET_TYPES.close_window, window.wid]) 1 PaintWorker.js:188 canvas transfer for window 22 : OffscreenCanvas {width: 123, height: 45, oncontextlost: null, oncontextrestored: null} false ```

[TijZwa]

Thanks! The issue while freezing is a throttle package not being handled the way it should. Throttle packages are queued when the VideoDecoder pipeline is not keeping up. VideoDecoder only works with chrome and secure context (localhost or SSL). Let me fix that.

As for the firefox issue: I can reproduce the issue.

[TijZwa]

@ToppDev Does PR #213 fixes the chrome issue?

[ToppDev]

Yes, the fix for chrome is working. The windows stay responsive

[TijZwa]

Cool. @totaam: Please check out the video. This filemanger is sending videoframes (as the throttle package only comes from VideoDecoder)

As for Firefox; the whole offscreen pipeline is changed since version 105. (https://www.mozilla.org/en-US/firefox/105.0/releasenotes/)

[totaam]

Support for the Offscreen Canvas DOM API with full context and font support. The OffscreenCanvas API provides a canvas that can be rendered off-screen in both Window and Web Worker contexts. I'm on Fedora 37 which has Firefox 107. @TijZwa does this mean that we need to disable offscreen decoding on Firefox? Or just for some versions?

As for thunar sending video frames, perhaps we need to add a text file mapping to https://github.com/Xpra-org/xpra/blob/master/fs/etc/xpra/content-type/50_class.conf

[TijZwa]

@TijZwa does this mean that we need to disable offscreen decoding on Firefox? Or just for some versions?

Yes, that is the safest bet for now. I did this in PR #215 @ToppDev Firefox should not flicker anymore, but the performance might be worse than Chrome.

@totaam I think we should focus on moving the paint logic to WebGL instead of fixing this issue for Firefox.

[ToppDev]

I tried to test it, Firefox does not let me edit the files in the Web Developer Tools though. So I tried to manually edit the files on my server under e.g. /usr/share/xpra/www/js/OffscreenDecodeWorker.js and restarting the xpra and nginx service but my browser still gets the old file versions somehow. (probably the worst approach to test it, but I am not really a web developer)

However, thanks for fixing. I will just wait till the fixes make it into the next release.

[totaam]

@ToppDev you can just wipe /usr/share/xpra/www and copy all the new html5 source tree there from a fresh git checkout. (restarting xpra or nginx is a good idea)

[ToppDev]

@totaam thank you very much, that worked! And also thank you @TijZwa again for fixing this so fast.