search

Xpra-org / xpra

Persistent remote applications for X11; screen sharing for X11, MacOS and MSWindows.
https://xpra.org/
GNU General Public License v2.0
1.91k stars 164 forks source link

xpra 4.1: blurry overlay in intellij #3042

Closed kellerkindt closed 3 years ago

kellerkindt commented 3 years ago

When I upgraded my client a few minutes ago to 4.1-r28561-1, I immediately noticed this in intellij image

xpra attach --env=XPRA_CLIPBOARD_IMAGE_STAMP=0 --env=XPRA_MODAL_WINDOWS=1 --ssh "ssh -p 22 " ssh://me@the-machine:22 --video-encoders=nvenc --encoding=png --compressors=lz4 --compress=0 --video-scaling=off --speaker=off --desktop-scaling=off

When I switch to the rgb encoding, it becomes better and does not happen all the time.

FYI, I run cinnamon on both computers and both use 0.9 for the font scaling factor, in the case this should make any difference.

totaam commented 3 years ago

Can you grab xpra info when this window is up?

Intellij is an IDE so no matter how badly it repaints its windows (a common occurence), it should always be sent as text (lossless) - unfortunately, no one ever bothered to submit non open-source application like intellij, so those are not detected properly.

--env=XPRA_MODAL_WINDOWS=1

This is better: --modal-windows=true.

--video-encoders=nvenc

Don't do that. Using nvenc alone is not a good idea. Its startup time is slow, xpra needs alternatives to operate smoothly.

--encoding=png

Again, I'm not sure what you're trying to achieve here. But this is generally a bad idea. If you want lossless screen updates, raise the min-quality instead of second guessing the best encoding.

--compressors=lz4

This is also a bad idea. Just leave the packet compressors on default settings.

--compress=0

I'm not sure what you're trying to achieve with this. Best to leave it alone.

kellerkindt commented 3 years ago

I dont feel comfortable dumping the whole log. Are there any sections you are particularly interested in? I guess window 5 is the main IntelliJ / CLion window, and window 28 is the popup-window:

windows.5.XShm=True
windows.5.above=False
windows.5.allowed-actions=('_NET_WM_ACTION_CLOSE', '_NET_WM_ACTION_MOVE', '_NET_WM_ACTION_RESIZE', '_NET_WM_ACTION_FULLSCREEN', '_NET_WM_ACTION_MINIMIZE', '_NET_WM_ACTION_SHADE', '_NET_WM_ACTION_STICK', '_NET_WM_ACTION_MAXIMIZE_HORZ', '_NET_WM_ACTION_MAXIMIZE_VERT', '_NET_WM_ACTION_CHANGE_DESKTOP', '_NET_WM_ACTION_ABOVE', '_NET_WM_ACTION_BELOW')
windows.5.below=False
windows.5.bypass-compositor=0
windows.5.children=((2097296, 0, -26, 1920, 1316, 0, 24),)
windows.5.class-instance=('jetbrains-clion', 'jetbrains-clion')
windows.5.client-geometry=(2869, 97, 1920, 1290)
windows.5.client-machine=pc973l
windows.5.command=
windows.5.content-type=
windows.5.decorations=1
windows.5.depth=24
windows.5.focused=True
windows.5.fullscreen=False
windows.5.geometry=(2869, 97, 1920, 1290)
windows.5.grabbed=0
windows.5.has-alpha=False
windows.5.icon-title=rust – src/model/mod.rs
windows.5.iconic=False
windows.5.maximized=False
windows.5.modal=False
windows.5.opacity=-1
windows.5.override-redirect=False
windows.5.pid=4985
windows.5.set-initial-position=True
windows.5.shaded=False
windows.5.shown=True
windows.5.size=(1920, 1290)
windows.5.size-constraints.gravity=1
windows.5.size-constraints.minimum-size=(340, 0)
windows.5.size-constraints.position=(6360, 170)
windows.5.skip-pager=False
windows.5.skip-taskbar=False
windows.5.state=('_NET_WM_STATE_FOCUSED',)
windows.5.sticky=False
windows.5.title=rust – src/model/mod.rs
windows.5.tray=False
windows.5.window-type=('NORMAL',)
windows.5.workspace=1
windows.5.xid=0x20008d

windows.28.XShm=True
windows.28.above=False
windows.28.allowed-actions=('_NET_WM_ACTION_CLOSE', '_NET_WM_ACTION_MOVE', '_NET_WM_ACTION_RESIZE', '_NET_WM_ACTION_FULLSCREEN', '_NET_WM_ACTION_MINIMIZE', '_NET_WM_ACTION_SHADE', '_NET_WM_ACTION_STICK', '_NET_WM_ACTION_MAXIMIZE_HORZ', '_NET_WM_ACTION_MAXIMIZE_VERT', '_NET_WM_ACTION_CHANGE_DESKTOP', '_NET_WM_ACTION_ABOVE', '_NET_WM_ACTION_BELOW')
windows.28.below=False
windows.28.bypass-compositor=0
windows.28.class-instance=('jetbrains-clion', 'jetbrains-clion')
windows.28.client-machine=pc973l
windows.28.command=
windows.28.content-type=
windows.28.depth=24
windows.28.focused=False
windows.28.fullscreen=False
windows.28.geometry=(3162, 513, 833, 578)
windows.28.grabbed=0
windows.28.has-alpha=False
windows.28.maximized=False
windows.28.opacity=-1
windows.28.override-redirect=True
windows.28.pid=4985
windows.28.shaded=False
windows.28.shown=True
windows.28.size=(833, 578)
windows.28.skip-pager=False
windows.28.skip-taskbar=True
windows.28.state=('_NET_WM_STATE_SKIP_TASKBAR',)
windows.28.sticky=False
windows.28.title=win4
windows.28.transient-for=5
windows.28.tray=False
windows.28.window-type=('DIALOG',)
windows.28.workspace=65535
windows.28.xid=0x200301

Regarding the silly startup parameters: They are the result of me fiddling around until I the quality and latency felt fine and consistently, at which I stopped changing them ever since.

totaam commented 3 years ago

Are there any sections you are particularly interested in?

Yes, the latency bits may explain why xpra decided to drop the quality so much.

windows.5.class-instance=('jetbrains-clion', 'jetbrains-clion') .. windows.28.class-instance=('jetbrains-clion', 'jetbrains-clion')

c446771dbc17a83a08e49831425e7987647da8f4 will now handle those windows as text (always lossless). That won't fix the underlying issue, but will fix this particular application. You can apply the one-liner change to your installation.

kellerkindt commented 3 years ago

Here you go, grep -i latency /tmp/xpra.info:

client.connection.client.ping_latency.50p=2
client.connection.client.ping_latency.80p=2
client.connection.client.ping_latency.90p=2
client.connection.client.ping_latency.avg=1
client.connection.client.ping_latency.cur=2
client.connection.client.ping_latency.max=3
client.connection.client.ping_latency.min=1
client.connection.latency.absmin=0
client.connection.server.ping_latency.50p=2
client.connection.server.ping_latency.80p=3
client.connection.server.ping_latency.90p=17
client.connection.server.ping_latency.avg=3
client.connection.server.ping_latency.cur=1
client.connection.server.ping_latency.max=17
client.connection.server.ping_latency.min=1
client.damage.client-latency=58940
client.damage.frame-total-latency=58940
client.damage.in_latency.90p=1127
client.damage.in_latency.avg=248
client.damage.in_latency.cur=0
client.damage.in_latency.max=1344
client.damage.in_latency.min=0
client.damage.out_latency.90p=1128
client.damage.out_latency.avg=248
client.damage.out_latency.cur=1
client.damage.out_latency.max=1346
client.damage.out_latency.min=0
client.window.5.batch.client-latency=(65, 21)
client.window.5.batch.client-latency.aim=800
client.window.5.batch.client-latency.aimed_avg=433
client.window.5.batch.client-latency.avg=17
client.window.5.batch.client-latency.div=1000
client.window.5.batch.client-latency.recent=7
client.window.5.batch.client-latency.smoothing=sqrt
client.window.5.batch.client-latency.target=5
client.window.5.batch.client-latency.weight_multiplier=365
client.window.5.batch.client-ping-latency=(49, 12)
client.window.5.batch.client-ping-latency.aim=949
client.window.5.batch.client-ping-latency.aimed_avg=240
client.window.5.batch.client-ping-latency.avg=1
client.window.5.batch.client-ping-latency.div=1000
client.window.5.batch.client-ping-latency.recent=1
client.window.5.batch.client-ping-latency.smoothing=sqrt
client.window.5.batch.client-ping-latency.target=6
client.window.5.batch.client-ping-latency.weight_multiplier=179
client.window.5.batch.server-ping-latency=(87, 18)
client.window.5.batch.server-ping-latency.aim=949
client.window.5.batch.server-ping-latency.aimed_avg=765
client.window.5.batch.server-ping-latency.avg=17
client.window.5.batch.server-ping-latency.div=1000
client.window.5.batch.server-ping-latency.recent=17
client.window.5.batch.server-ping-latency.smoothing=sqrt
client.window.5.batch.server-ping-latency.target=21
client.window.5.batch.server-ping-latency.weight_multiplier=526
client.window.5.damage.in_latency.90p=56
client.window.5.damage.in_latency.avg=37
client.window.5.damage.in_latency.cur=0
client.window.5.damage.in_latency.max=488
client.window.5.damage.in_latency.min=0
client.window.5.damage.out_latency.90p=56
client.window.5.damage.out_latency.avg=38
client.window.5.damage.out_latency.cur=0
client.window.5.damage.out_latency.max=490
client.window.5.damage.out_latency.min=0
client.window.5.damage.target-latency=14
client.window.5.encoding.speed.factors.damage-latency-abs=0
client.window.5.encoding.speed.factors.damage-latency-rel=26
client.window.5.encoding.speed.factors.decoding-latency=1
client.window.5.encoding.speed.limits.damage-latency=256

client.window.28.batch.client-latency=(42, 27)
client.window.28.batch.client-latency.aim=800
client.window.28.batch.client-latency.aimed_avg=183
client.window.28.batch.client-latency.avg=13
client.window.28.batch.client-latency.div=1000
client.window.28.batch.client-latency.recent=2
client.window.28.batch.client-latency.smoothing=sqrt
client.window.28.batch.client-latency.target=5
client.window.28.batch.client-latency.weight_multiplier=365
client.window.28.batch.client-ping-latency=(52, 11)
client.window.28.batch.client-ping-latency.aim=949
client.window.28.batch.client-ping-latency.aimed_avg=275
client.window.28.batch.client-ping-latency.avg=1
client.window.28.batch.client-ping-latency.div=1000
client.window.28.batch.client-ping-latency.recent=1
client.window.28.batch.client-ping-latency.smoothing=sqrt
client.window.28.batch.client-ping-latency.target=6
client.window.28.batch.client-ping-latency.weight_multiplier=173
client.window.28.batch.server-ping-latency=(52, 11)
client.window.28.batch.server-ping-latency.aim=949
client.window.28.batch.server-ping-latency.aimed_avg=279
client.window.28.batch.server-ping-latency.avg=4
client.window.28.batch.server-ping-latency.div=1000
client.window.28.batch.server-ping-latency.recent=2
client.window.28.batch.server-ping-latency.smoothing=sqrt
client.window.28.batch.server-ping-latency.target=6
client.window.28.batch.server-ping-latency.weight_multiplier=163
client.window.28.damage.in_latency.90p=0
client.window.28.damage.in_latency.avg=1
client.window.28.damage.in_latency.cur=0
client.window.28.damage.in_latency.max=37
client.window.28.damage.in_latency.min=0
client.window.28.damage.out_latency.90p=1
client.window.28.damage.out_latency.avg=1
client.window.28.damage.out_latency.cur=1
client.window.28.damage.out_latency.max=38
client.window.28.damage.out_latency.min=0
client.window.28.damage.target-latency=12
client.window.28.encoding.speed.factors.damage-latency-abs=0
client.window.28.encoding.speed.factors.damage-latency-rel=0
client.window.28.encoding.speed.factors.decoding-latency=1
client.window.28.encoding.speed.limits.damage-latency=170
network.system.ipv4.tcp.low_latency=True

EDIT:

FYI 

windows.32.class-instance=('code', 'Code')
windows.32.title=xpra-latency.info - Visual Studio Code
totaam commented 3 years ago

windows.32.class-instance=('code', 'Code')

Mapped to 'text' in bd4a3c63b0b3c125676899c972d291824a9648a5.

client.damage.frame-total-latency=58940

That's insanely high. AFAICT, this can only happen if your network is having serious issues.

kellerkindt commented 3 years ago

The two computers are connected via gigabit local ethernet and it runs with stable and smooth latency between 70ms or 380ms (depending on the encoding). I can only assume that intellij causes a short spike here (the popup shows all classes / files matching the entered pattern and is opened via a keyboard shortcut, so it might induce a very short heavy load?)

totaam commented 3 years ago

I can only assume that intellij causes a short spike here

This is a weighted average, and unless the maths are wrong - it comes up with a value of 58940 milliseconds, which is nearly 1 minute!

kellerkindt commented 3 years ago

Well, that seems to be wrong, then o.o Could it be, that this average is wrong because the window / popup has just been opened? I would like to send you the graphs to proof that it behaves very well normally, but I wont have access to these machines for the next ~8 hours.

As a side note: the popup sometimes starts to open on the wrong monitor until I move or resize the main window, but that seems to be an independent issue?

allo- commented 3 years ago

I've got a similar problem with PNG/P encoding and Firefox Bookmark overlay (the small overlay that pops up when you click the star icon in the address bar) and extension overlays like the one from CookieAutodelete. With h264 encoding it works for me.

totaam commented 3 years ago

I've got a similar problem with PNG/P encoding

Why would anyone choose PNG/P knowingly? Have you upgraded to 4.1.2?

and Firefox Bookmark overlay

Can you capture the xpra info output when this popup window is shown blurry? Here's what I have for the "star icon" one:

windows.7.above=False
windows.7.allowed-actions=('_NET_WM_ACTION_CLOSE', '_NET_WM_ACTION_MOVE', '_NET_WM_ACTION_RESIZE', '_NET_WM_ACTION_FULLSCREEN', '_NET_WM_ACTION_MINIMIZE', '_NET_WM_ACTION_SHADE', '_NET_WM_ACTION_STICK', '_NET_WM_ACTION_MAXIMIZE_HORZ', '_NET_WM_ACTION_MAXIMIZE_VERT', '_NET_WM_ACTION_CHANGE_DESKTOP', '_NET_WM_ACTION_ABOVE', '_NET_WM_ACTION_BELOW')
windows.7.below=False
windows.7.bypass-compositor=2
windows.7.class-instance=('Firefox', 'Firefox')
windows.7.client-machine=localhost.localdomain
windows.7.command=
windows.7.content-type=browser
windows.7.depth=32
windows.7.focused=False
windows.7.fullscreen=False
windows.7.geometry=(3867, 446, 349, 560)
windows.7.grabbed=0
windows.7.has-alpha=True
windows.7.maximized=False
windows.7.opacity=-1
windows.7.override-redirect=True
windows.7.pid=259280
windows.7.role=Popup
windows.7.shaded=False
windows.7.shown=True
windows.7.size=(349, 560)
windows.7.skip-pager=False
windows.7.skip-taskbar=False
windows.7.state=()
windows.7.sticky=False
windows.7.title=Firefox
windows.7.transient-for=2
windows.7.tray=False
windows.7.window-type=('UTILITY',)
windows.7.workspace=65535
windows.7.xid=0x800034

The bit that I am interested in will be prefixed with client.window.$WINDOWID.

With h264 encoding it works for me.

Do you mean auto?

totaam commented 3 years ago

This change: 4eb311afe40f43b2e3cf48910c42adbdf33d7935 will paper over the issue for some window types, including the UTILITY window in the Firefox example above.

But to properly fix this blurriness issue, I will either have to be able to reproduce it here or get the detailed xpra info data showing why the xpra server engine thinks it needs to reduce the quality.

allo- commented 3 years ago

Why would anyone choose PNG/P knowingly?

Sharp fonts and less traffic than PNG. It seems a good compromise for me. h264/vp8 are faster but I get blurry fonts and changing colors for fonts.

Have you upgraded to 4.1.2?

Not yet, because I backported the "upgrade" patch myself and didn't want to overwrite my changes when I may expect a version containing the patch soon.

Can you capture the xpra info output when this popup window is shown blurry?

I'll take a look after an upgrade.

totaam commented 3 years ago

Sharp fonts and less traffic than PNG...

As per https://github.com/Xpra-org/xpra/blob/master/docs/Usage/Encodings.md: Generally, if any tuning is needed, instead of trying to guess what should be used and overriding the encodings and encoding options, it is best to use the min-speed and min-quality options instead PNG/P is a worse option than setting quality=100

allo- commented 3 years ago

My experiments with different codecs worked best with png/P for mostly static content.

I am not sure about the bandwidth/quality compromise, but I hate having unsharp edges on text or flickering between similar but different colors when scrolling with the different video codecs.

VP9 worked best, but it still had some minor artifacts and when reading text I notice such artifacts.

totaam commented 3 years ago

My experiments with different codecs worked best with png/P for mostly static content.

As I said above and in the documentation and at every opportunity: don't play with codecs, change the quality instead.

totaam commented 3 years ago

@kellerkindt Is the overlay still blurry with 4.2.1?

When it comes to tuning the encodings, 2 new data points:

kellerkindt commented 3 years ago

Seems like the blurryness is fixed with xpra v4.2.1-r0.

But as we as it is related: the window does (and already has) sometimes open on the wrong monitor. Intellij is running on monitor 2, I press the shortcut for the overlay window of listing all occurrences, which opens the overlay window on monitor 3 instead of monitor 2. But it seems, that its about the right position where I would expect it... on the other monitor...

When I drag the main window of IntelliJ a bit back and forward, the overlay opens on the right monitor again... at least for a while.

stdedos commented 3 years ago

I think you are talking about https://github.com/Xpra-org/xpra/issues/2703 ?

kellerkindt commented 3 years ago

Thx, it seems so. Looks like its going to be fixed in the next version :D

stdedos commented 3 years ago

If you are talking about https://github.com/Xpra-org/xpra/issues/3202, it's fixed https://github.com/Xpra-org/xpra/issues/2703, however, is not.

stevenlafl commented 3 years ago

Can you grab xpra info when this window is up?

Intellij is an IDE so no matter how badly it repaints its windows (a common occurence), it should always be sent as text (lossless) - unfortunately, no one ever bothered to submit non open-source application like intellij, so those are not detected properly.

--env=XPRA_MODAL_WINDOWS=1

This is better: --modal-windows=true.

--video-encoders=nvenc

Don't do that. Using nvenc alone is not a good idea. Its startup time is slow, xpra needs alternatives to operate smoothly.

--encoding=png

Again, I'm not sure what you're trying to achieve here. But this is generally a bad idea. If you want lossless screen updates, raise the min-quality instead of second guessing the best encoding.

--compressors=lz4

This is also a bad idea. Just leave the packet compressors on default settings.

--compress=0

I'm not sure what you're trying to achieve with this. Best to leave it alone.

On 4.1.1 the nvenc is absolutely necessary, no other options placed on the commandline or via the xpra.conf file decrease compression or increase quality.

stdedos commented 3 years ago

unfortunately, no one ever bothered to submit non open-source application like intellij, so those are not detected properly.

Captain Right here:

totaam commented 3 years ago

--compress=0

I'm not sure what you're trying to achieve with this. Best to leave it alone.

On 4.1.1 the nvenc is absolutely necessary, no other options placed on the commandline or via the xpra.conf file decrease compression or increase quality.

@stevenlafl The compress option has absolutely nothing to do whatsoever with nvenc or any other pixel compression encoders. It is for data packets and should be left alone.