search

Xpra-org / xpra

Persistent remote applications for X11; screen sharing for X11, MacOS and MSWindows.
https://xpra.org/
GNU General Public License v2.0
1.9k stars 164 forks source link

shadow desktop ceases to update when using `--desktop-scaling` < 100% #4323

Open Lanchon opened 1 month ago

Lanchon commented 1 month ago

Describe the bug shadow desktop ceases to update a couple of seconds after starting when using --desktop-scaling < 100%. this happens regardless of whether OpenGL is used. mouse actions in the client to server direction keep on working. no text is displayed in the client console when the image ceases to update.

To Reproduce Steps to reproduce the behavior:

  1. server command: none
  2. client command:
    • xpra shadow ssh:XXX:0 --desktop-scaling=70%
    • xpra shadow ssh:XXX:0 --desktop-scaling=70% --opengl=off
  3. specific action to trigger the bug: wait 2 seconds

System Information (please complete the following information):

Additional context on both sides: Desktop: Cinnamon v: 6.2.9 tk: GTK v: 3.24.41 wm: Muffin vt: 7 dm: LightDM v: 1.30.0 standard X11 session

totaam commented 1 month ago

That's odd, is there anything in your client output at all? Your server log perhaps?

I did hit some major painting issues with opengl, but none without. The opengl issues are now fixed (and have already been backported to 6.1.x and will be included in the upcoming v6.1.2 release) I've just tried it here with Fedora 40, xpra from git master and everything worked as expected - attaching locally with --no-mmap.

Lanchon commented 4 weeks ago

i downgraded only the client, then tested several client versions against 6.1.1-r0:

(freezes = stops updating the client windows a couple of seconds after launching) (works = no freezing)

version opengl off opengl default
5.0.9-r0-1 works (bilinear interpolation) works (nearest neighbor)
6.0-r0-1 works works but with #4324 and significant dirty refresh issues
6.0.2-r0-1 freezes freezes and has #4324

the screen freezes exactly before the "xpra shadow server ready" notification is shown in the server. (note that client mouse and keyboard events continue to be forwarded to the sever normally.)

totaam commented 4 weeks ago

6.0.x are not supported versions: https://github.com/Xpra-org/xpra/wiki/Versions

significant dirty refresh issues

Partial workaround: https://github.com/Xpra-org/xpra/issues/4319#issuecomment-2281899406

Lanchon commented 4 weeks ago

well i just wanted to do a gross bisection regarding this freezing issue.

it was introduced between 6.0 and 6.0.2.

totaam commented 4 weeks ago

Thanks, I'm not seeing any freezing here - but I'll try again.

Lanchon commented 4 weeks ago

try linux mint 22 cinnamon, default session. im using that on both sides.

totaam commented 4 weeks ago

5.0.9-r0-1 works (bilinear interpolation) works (nearest neighbor)

Oh, and BTW, I don't see how that's possible unless your client display is also using some quite odd scaling settings. The opengl renderer uses linear: https://github.com/Xpra-org/xpra/issues/4324#issuecomment-2288029609

Lanchon commented 4 weeks ago

yes, technically it's actually called bilinear, given that it acts on a 2D surface and has to interpolate linearly in 2 axes, but we are talking about the same thing.

totaam commented 4 weeks ago

technically it's actually called bilinear

What is surprising here is that you're saying that the non-opengl client uses linear and the opengl client does not. Both should be using server-side downscaling, which means that the client-side rendering should be irrelevant - and identical.

totaam commented 4 weeks ago

Found a major issue with opengl scaling, which I still don't fully understand but is finally fixed: 775d435b83097b0d97d55b3173ab5d9503ee5971

I believe that this is the same issue that you reported here as it requires desktop-scaling to trigger. The updates are being sent to the client, but the opengl client was somehow failing to update the window front buffer.

Feel free to re-open if 6.1.2 doesn't fix things for you. (there will be 6.2 builds in the beta area soon too)

Lanchon commented 4 weeks ago

What is surprising here is that you're saying that the non-opengl client uses linear and the opengl client does not. Both should be using server-side downscaling, which means that the client-side rendering should be irrelevant - and identical.

that is correct.

i hope you don't need to see this with your own eyes, i have tested it repeatedly. i also saw that opengl mode was more power efficient on the client so i wanted to use that, but i couldn't because scaled down text is poorly readable without proper filtering.

keep in mind that this is a 5.x client talking to a 6.x server, so it's probably a setup you don't care much to debug.

Both should be using server-side downscaling

how does the 6.x server do the downscaling? this is an operation that would benefit from GPU usage. even if today's CPUs can do it, it'd be much more efficient to do it in the GPU. this also applies to image/video encoding of course. does the server use the GPU? can i tell the server NOT to use the GPU? from the client? are you sure --opengl=off does not influence the server?

Lanchon commented 4 weeks ago

Feel free to re-open

i can't reopen if a team member closed the issue; only you can. (i could reopen it if i had closed it myself though.)

Found a major issue with opengl scaling, which I still don't fully understand but is finally fixed: 775d435

I believe that this is the same issue that you reported here as it requires desktop-scaling to trigger. The updates are being sent to the client, but the opengl client was somehow failing to update the window front buffer.

according to your description, it is not the same bug. unfortunately, as stated in the OP, this issue happens during downscaling even if opengl is disabled in the client. so you can go ahead and reopen this, sorry...

totaam commented 4 weeks ago

how does the 6.x server do the downscaling?

Typically via libyuv: https://github.com/Xpra-org/xpra/blob/4a5f14e80cdf5de843ac0ae7a3c17a72ecadf16e/xpra/codecs/libyuv/converter.pyx#L623

this is an operation that would benefit from GPU usage

Very much so.

even if today's CPUs can do it, it'd be much more efficient to do it in the GPU.

Not always. The pixel data usually resides in CPU memory. Also, setting up a GPU context can be expensive, it isn't worth it for small sizes.

this also applies to image/video encoding of course. does the server use the GPU?

Yes, if available. In that case, it will upload (if needed) the pixel data and do colourspace conversion, downscaling and encoding on the GPU.

can i tell the server NOT to use the GPU?

Sure: --video-encoders=x264,vpx

from the client?

No. The client has its own switches for GPU decoding.

are you sure --opengl=off does not influence the server?

Yes.

Anyway, that turned out to be interesting, as I found lots of small issues along the way:

And the bilinear vs nearest sampling issue is now tracked here: #4326

The bad news is that apart from these issues, I left my connection to the shadow server open for hours and did not experience any other glitches. (and that's to and from Linux Mint 22 - except for the OpenGL accelerated client which cannot run in VirtualBox, so I used Fedora 40 to connect)

Lanchon commented 4 weeks ago

The bad news is that apart from these issues, I left my connection to the shadow server open for hours and did not experience any other glitches.

you didn't need to. this bug is extremely reliable, happening 100% of the time and before the "xpra shadow server ready" notification (so, about 2 seconds after starting showing the remote display).

EDIT: disregard all that comes next, i got mixed up :(

can i tell the server NOT to use the GPU?

Sure: --video-encoders=x264,vpx

from the client?

No.

so the difference between your case and mine seems to be that my server is using the integrated intel GPU and yours is not. which points to they issue probably being worked-around by turning off GPU use in the server.

1) is this assessment compatible with the fact that changing the client from 6.x to 5.x also sidesteps the issue? (this requires that using a 5.x client alters the codepaths related to GPU use in the 6.x server.)

2) what can i add to client command xpra shadow ssh:XXX:0 --desktop-scaling=70% --opengl=off to disable GPU usage in the server so i can test the hypothesis?

totaam commented 3 weeks ago

so the difference between your case and mine seems to be that my server is using the integrated intel GPU and yours is not.

I have no idea where you got this from. Did you try to force the video encoders as per the previous comment? Did it help?

which points to they issue probably being worked-around by turning off GPU use in the server.

How so?

is this assessment compatible with the fact that changing the client from 6.x to 5.x also sidesteps the issue?

No, it is not.

what can i add to client command

Start your server independently via an ssh session and use the --video-encoders=x264,vpx switch. Then connect your client as a separate command.

Lanchon commented 3 weeks ago

so the difference between your case and mine seems to be that my server is using the integrated intel GPU and yours is not.

I have no idea where you got this from.

sorry my bad, in my head i mixed the the bug report regarding start-desktop and this one regarding shadow desktop. so please disregard the previous message.

anyway...

on both client and server running v6.1.1-r0 on mint 22, as per the OP, trying this command fails because the screen stops refreshing, as described in the OP:

xpra shadow --opengl=off --desktop-scaling=70% ssh:XXX:0

however if i do this, the screen keeps on refreshing! the bug never shows up... weird:

ssh XXX xpra shadow :0
xpra attach --opengl=off --desktop-scaling=70% ssh:XXX:0
...
xpra stop ssh:XXX:0

so... there seems to be a race condition. take this command:

ssh XXX xpra shadow :0; sleep N; xpra attach --opengl=off --desktop-scaling=70% ssh:XXX:0; xpra stop ssh:XXX:0

in my case, if N >= 5 then the connection works fine. but if N is smaller it tends to stop refreshing the display. the stopping happens exactly as the "xpra shadow server ready" notification is shown in the server, as said earlier. so it's just a question of not connecting before the notification happens.

furthermore, if you don't wait and the connection freezes, simply disconnecting and attaching again fixes the issue. (of course this doesn't work with the xpra shadow ssh:XXX:0-style commands, as they tear the server down immediately on disconnection.)

Did you try to force the video encoders as per the previous comment? Did it help?

so if you add --video-encoders=x264,vpx and remove the sleep, it works fine:

ssh XXX xpra shadow --video-encoders=x264,vpx :0; xpra attach --opengl=off --desktop-scaling=70% ssh:XXX:0; xpra stop ssh:XXX:0

no delay is needed which means the race condition is gone.

maybe consider adding something like --server-option="--video-encoders=x264,vpx" to allow passing arguments to the server.

totaam commented 3 weeks ago

so if you add --video-encoders=x264,vpx and remove the sleep, it works fine:

So the problem is very likely to be related to gstreamer or some other hardware accelerated encoder. v6.2.x has a --no-gstreamer command line option for this purpose: #4330

maybe consider adding something like --server-option="--video-encoders=x264,vpx" to allow passing arguments to the server.

Most command line options are already forwarded to the server automatically, without the need for a server-option, unless they are listed in: https://github.com/Xpra-org/xpra/blob/80a3ba3116b3db37d612879639b097c73a300e53/xpra/scripts/config.py#L786

The problem with encoding options (like video-encoders, etc) is that the command line parsing will replace the defaults (ie: all) with the actual values to be used (ie: h264, vp8, ...) but this substitution is only valid on the client, as the server may have completely different modules installed. Changing this is very much non-trivial.