search

neutrinolabs / xrdp

xrdp: an open source RDP server
http://www.xrdp.org/
Apache License 2.0
5.65k stars 1.73k forks source link

protocol error with very wide screen #2063

Closed rowlap closed 7 months ago

rowlap commented 2 years ago

There seems to be a problem with screen layouts exceeding 8192 pixels wide overall, for example 3x 3840x2160 monitors in a row.

If the connection parameters are chosen to obtain RemoteFX (i.e. 32-bit color depth, LAN mode) then after successful authentication, mstsc.exe (Windows 10 20H2) quits with the message

Because of a protocol error, this session will be disconnected. Please try connecting to the remote computer again.

Nothing stands out in xrdp / sesman debug logs as being a protocol error when mstsc disconnects.

If I request a specific height/width on the commandline, both xrdp and Windows servers provide max 8192 in "windowed mode", but Windows-to-Windows does work with Full Screen "Use all my monitors".

matt335672 commented 2 years ago

Thanks for reporting this.

May I ask what is the output of xrdp -v? This will tell us the flags xrdp is compiled with.

The RDP protocol has limits of 8192 x 8192 for any one monitor, and 32766 x 32766 for the overall desktop. See here and also [MS-RDPBCGR] Appendix A notes <45> and <46>

rowlap commented 2 years ago 
$ xrdp -v
xrdp 0.9.17
  A Remote Desktop Protocol Server.
  Copyright (C) 2004-2020 Jay Sorg, Neutrino Labs, and all contributors.
  See https://github.com/neutrinolabs/xrdp for more information.

  Configure options:
      --build=x86_64-redhat-linux-gnu
      --host=x86_64-redhat-linux-gnu
      --program-prefix=
      --disable-dependency-tracking
      --prefix=/usr
      --exec-prefix=/usr
      --bindir=/usr/bin
      --sbindir=/usr/sbin
      --sysconfdir=/etc
      --datadir=/usr/share
      --includedir=/usr/include
      --libdir=/usr/lib64
      --libexecdir=/usr/libexec
      --localstatedir=/var
      --sharedstatedir=/var/lib
      --mandir=/usr/share/man
      --infodir=/usr/share/info
      --enable-pixman
      --enable-painter
      --enable-vsock
      --with-socketdir=/run/xrdp
      build_alias=x86_64-redhat-linux-gnu
      host_alias=x86_64-redhat-linux-gnu
      CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1  -m64 -mtune=generic
      LDFLAGS=-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
      PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig

  Compiled with OpenSSL 1.0.2k-fips  26 Jan 2017

I appreciate that 3x 4k monitors are not a typical screen setup, but it's consistently reproducible with the right hardware. The per-monitor limits should be OK (and indeed it works during the xrdp login prompt).

matt335672 commented 2 years ago

This is likely to be a duplicate of #2068

Once we've verified a patch for that we can see if it indeed fixes this one too.

matt335672 commented 2 years ago

@rowlap - we've released v0.9.18 which may resolve this.

Are you in a position to try this release, or are you depending on a particular distro to package it for you?

Thanks.

rowlap commented 2 years ago

Looks like https://src.fedoraproject.org/rpms/xrdp already has the new release in testing (and it's easier to test with those packages).

matt335672 commented 2 years ago

Yes, you're quite right.

There's a small chance the RHEL 7/8 ones may get replaced with updates for #2118, but these won't affect your platform.

rowlap commented 2 years ago

Tested against 0.9.18-5 Fedora RPM, but the same output is seen from mstsc "Because of a protocol error, this session will be disconnected."

matt335672 commented 2 years ago

OK - thanks.

labmonkey42 commented 2 years ago

Chiming in with an Ubuntu 22.04 VM doing this when I attempt to use all three 4k monitors (horizontally aligned).

xrdp 0.9.17
  A Remote Desktop Protocol Server.
  Copyright (C) 2004-2020 Jay Sorg, Neutrino Labs, and all contributors.
  See https://github.com/neutrinolabs/xrdp for more information.

  Configure options:
      --enable-ipv6
      --enable-jpeg
      --enable-fuse
      --enable-rfxcodec
      --enable-opus
      --enable-painter
      --enable-vsock
      --build=x86_64-linux-gnu
      --prefix=/usr
      --includedir=${prefix}/include
      --mandir=${prefix}/share/man
      --infodir=${prefix}/share/info
      --sysconfdir=/etc
      --localstatedir=/var
      --disable-silent-rules
      --libdir=${prefix}/lib/x86_64-linux-gnu
      --libexecdir=${prefix}/lib/x86_64-linux-gnu
      --disable-maintainer-mode
      --disable-dependency-tracking
      --with-socketdir=/run/xrdp/sockdir
      build_alias=x86_64-linux-gnu
      CFLAGS=-g -O2 -ffile-prefix-map=/build/xrdp-asBei3/xrdp-0.9.17=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security
      LDFLAGS=-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now -Wl,--as-needed
      CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2 -Wno-error=deprecated-declarations
      PKG_CONFIG_PATH=/build/xrdp-asBei3/xrdp-0.9.17/pkgconfig

  Compiled with OpenSSL 3.0.2 15 Mar 2022
metalefty commented 2 years ago

8192x8192 per monitor is the limit when using RemotFX. Just a guess, xrdp (librfxcodec) might process 3 monitors as 1 monitor.

metalefty commented 1 year ago

I could reproduce the issue at fafd7bf

metalefty commented 1 year ago 
[20221007-14:54:50] [ERROR] xrdp_sec_send_fastpath: xrdp_fastpath_send failed
[20221007-14:54:50] [ERROR] xrdp_rdp_send_fastpath: xrdp_sec_send_fastpath failed
[20221007-14:54:50] [ERROR] libxrdp_fastpath_send_frame_marker: xrdp_rdp_send_fastpath failed
[20221007-14:54:50] [ERROR] xrdp_iso_send: trans_write_copy_s failed
[20221007-14:54:50] [DEBUG] Closed socket 12 ([::ffff:192.168.24.5]:3389)

When I turned off fast path, the protocol error doesn't occur but screen is not drawn. Something is going wrong in screen encoding but RDP session is alive and other things are wokring (keyboard, mouse input, sound redirection). It works like that I'm operating computer with eyes closed.

There might be two issues:

metalefty commented 8 months ago

It will be solved by GFX we're working at #2891. No protocol errors 3840+1920+3840 pixels monitor.

300024848-c5f4d7ec-366d-4ed6-80e8-35390020a908

metalefty commented 7 months ago

GFX has been merged into devel and v0.10, closing.

labmonkey42 commented 7 months ago

Confirming my Ubuntu 22.04.03 desktop with 11520x2160 (3x 4k horizontally arranged) is working now.

Edit: adding version info

xrdp 0.9.17
  A Remote Desktop Protocol Server.
  Copyright (C) 2004-2020 Jay Sorg, Neutrino Labs, and all contributors.
  See https://github.com/neutrinolabs/xrdp for more information.

  Configure options:
      --enable-ipv6
      --enable-jpeg
      --enable-fuse
      --enable-rfxcodec
      --enable-opus
      --enable-painter
      --enable-vsock
      --build=x86_64-linux-gnu
      --prefix=/usr
      --includedir=${prefix}/include
      --mandir=${prefix}/share/man
      --infodir=${prefix}/share/info
      --sysconfdir=/etc
      --localstatedir=/var
      --disable-silent-rules
      --libdir=${prefix}/lib/x86_64-linux-gnu
      --libexecdir=${prefix}/lib/x86_64-linux-gnu
      --disable-maintainer-mode
      --disable-dependency-tracking
      --with-socketdir=/run/xrdp/sockdir
      build_alias=x86_64-linux-gnu
      CFLAGS=-g -O2 -ffile-prefix-map=/build/xrdp-asBei3/xrdp-0.9.17=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security
      LDFLAGS=-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now -Wl,--as-needed
      CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2 -Wno-error=deprecated-declarations
      PKG_CONFIG_PATH=/build/xrdp-asBei3/xrdp-0.9.17/pkgconfig

  Compiled with OpenSSL 3.0.2 15 Mar 2022

Uhh, why is this working without using the new version?!