Unable to start Xpra (arm64) on m1

Xpra-org / xpra

Persistent remote applications for X11; screen sharing for X11, MacOS and MSWindows.

https://xpra.org/

GNU General Public License v2.0

1.94k stars 168 forks source link

Unable to start Xpra (arm64) on m1 #4017

Closed pefribeiro closed 3 months ago

pefribeiro commented 1 year ago

Describe the bug Unable to start Xpra on M1.

To Reproduce Steps to reproduce the behavior:

Open Xpra, eg. via command line

System Information (please complete the following information):

macOS Ventura 13.6
Latest publicly available dmg via https://xpra.org/beta/MacOS/arm64/Xpra.dmg (Note that the link in Installation section points to https://xpra.org/dists/MacOS/x86_64/Xpra.dmg, the x86_64 version. The .pkg versions have a typo in the URL.)

Additional context

grep: /usr/share/locale/locale.alias: No such file or directory
xpra main error:
Traceback (most recent call last):
  File "/Applications/Xpra.app/Contents/Resources/lib/python/xpra/scripts/main.py", line 123, in main
    return run_mode(script_file, cmdline, err, options, args, mode, defaults)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Applications/Xpra.app/Contents/Resources/lib/python/xpra/scripts/main.py", line 457, in run_mode
    return do_run_mode(script_file, cmdline, error_cb, options, args, mode, defaults)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Applications/Xpra.app/Contents/Resources/lib/python/xpra/scripts/main.py", line 589, in do_run_mode
    return gui.main(cmdline)
           ^^^^^^^^^^^^^^^^^
  File "/Applications/Xpra.app/Contents/Resources/lib/python/xpra/gtk_common/gui.py", line 262, in main
    from xpra.platform.darwin.gui import wait_for_open_handlers
  File "/Applications/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py", line 16, in <module>
    import objc                         #@UnresolvedImport
    ^^^^^^^^^^^
  File "objc/__init__.pyc", line 6, in <module>
  File "objc/_objc.pyc", line 14, in <module>
  File "objc/_objc.pyc", line 10, in __load
  File "imp.pyc", line 343, in load_dynamic
ImportError: dlopen(/Applications/Xpra.app/Contents/Resources/lib/python/lib-dynload/objc/_objc.so, 0x0002): symbol not found in flat namespace '_ffi_find_closure_for_code_np'

totaam commented 1 year ago

_ffi_find_closure_for_code_np

ffi problems again, see https://github.com/Xpra-org/gtk-osx-build/issues/27 and https://github.com/Xpra-org/gtk-osx-build/issues/22

FYI: I don't have direct access to M1 or M2 Apple hardware to test on so this may take a while. Please considering sponsoring the hardware.

pefribeiro commented 1 year ago

Thanks @totaam. How would I go about compiling it with --with-system-ffi ?

totaam commented 1 year ago

@pefribeiro https://github.com/Xpra-org/gtk-osx-build

puremourning commented 1 year ago

@totaam I'm seeing the same issue here with the apple silicon beta, and would like to try and help. I spent a good part of yesterday trying to build locally using the gtk-osx-build fork, but keep hitting problems (complication failures etc.)

To save your time, can you confirm that these instructions are right: https://github.com/Xpra-org/xpra/blob/master/docs/Build/MacOS.md and there are no changes required to ~/.config/jhbuildrc-custom ? I sort of guessed that changing architectures=["x86_64"] might be required (though changing it to arm64 resulted in build failure in pixman.

running on a almost-newly-installed macOS 13.6 with Xcode 15, commandlinetools installed. In order to get past the first build problems I had to do cpan install XML::Parser (this may be because homebrew perl was in my path)

Detailed results of trying to get it built. First with architectures=['x86_64']:

gtk-osx-setup.sh: ok
jhbuild update - this fails as the zlib tarball is returning 404 (I fixed it by changing the repo URL; see below for error)
jhbuild bootstrap-gtk-osx : fails configuring m4

checking for socklen_t... no
checking for socklen_t equivalent... configure: error: Cannot find a type to use in place of socklen_t
*** Error during phase configure of m4: ########## Error running /Users/ben/gtk/source/m4-1.4.19/configure --prefix /Users/ben/gtk/inst    *** [2/15]

Next with architectures=['arm64']:

using the following:

setup_sdk(target="12", sdk_version="12", architectures=["arm64"])

I get the following results:

gtk-osx-setup.sh - ok
jhbuild update - this fails as the zlib tarball is returning 404 (I fixed it by changing the repo URL)
jhbuild bootstrap-gtk-osx - succeeds \o/
jhbuild build - fails building pixman with a screenful of errors, seemingly about some unrecognised asm block. perhaps this isn't building arm ?

/Users/ben/gtk/source/pixman-0.42.2/pixman/pixman-arma64-neon-asm.S:657:1: note: while in macro instantiation
generate_composite_function_single_scanline pixman_composite_scanline_add_asm_neon, 32, 0, 32, FLAG_DST_READWRITE, 8, default_init, default_cleanup, pixman_composite_add_8_8_process_pixblock_head, pixman_composite_add_8_8_process_pixblock_tail, pixman_composite_add_8888_8888_process_pixblock_tail_head
^
<instantiation>:13:9: error: expected absolute expression
    .if elem_size==32
        ^
<instantiation>:18:5: note: while in macro instantiation
    pixldst 4, st1, 32, basereg, mem_operand, abits
    ^
<instantiation>:37:5: note: while in macro instantiation
    pixst chunk_size, dst_w_bpp, dst_w_basereg, DST_W
    ^
<instantiation>:37:1: note: while in macro instantiation
.irp chunk_size, 16, 8, 4, 2, 1
^
<instantiation>:124:5: note: while in macro instantiation
    process_trailing_pixels 0, 1, process_pixblock_head, process_pixblock_tail, process_pixblock_tail_head
    ^
<instantiation>:1:1: note: while in macro instantiation
generate_composite_function_scanline 0, x
^

zlib issue. the URL is 404, so I changed the repo url to https://distfiles.macports.org/zlib/ based on quick google search for the tarball name :)

curl --continue-at - -L https://www.zlib.net/zlib-1.2.13.tar.xz -o /Users/ben/gtk/source/pkgs/zlib-1.2.13.tar.xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   315  100   315    0     0    569      0 --:--:-- --:--:-- --:--:--   570
*** Error during phase checkout of zlib: file hash is incorrect (expected d14c38e313afc35a9a8760dadf26042f51ea0f5d154b0630a31da0540107fb98, got d5a89e26beae0bc03ad18a0b0d1d3d75f87c32047879d25da11970cb5c4662a3) *** [4/137]

totaam commented 1 year ago

this may be because homebrew perl was in my path

That's likely as I never saw perl errors here on any arch.

this fails as the zlib tarball is returning 404

Perhaps it moved since I used that URL to download the source? (IIRC, this is from gtk-osx-build upstream) Please submit a PR with the URL fixes. Not using macports.org URL.

fails building pixman with a screenful of errors

Looks like https://github.com/microsoft/vcpkg/issues/29168

puremourning commented 1 year ago

zlib

Done - https://github.com/Xpra-org/gtk-osx-build/pull/31

pixman

Good shout. Traced it through and there looks to be a workaround (disable neon), but also that it's fixed recently.

For now I will try the workaround --disable-arm-a64-neon --disable-arm-neon. If that works, perhaps I'll try upgrading pixman to whatever version fixes it.

puremourning commented 1 year ago

OK, now I am getting build failure with cairo, which is looking for a LZO header.

/Applications/Xcode.app/Contents/Developer/usr/bin/gcc -Iutil/cairo-script/libcairo-script-interpreter.2.dylib.p -Iutil/cairo-script -I../../../../gtk/source/cairo-1.17.8/util/cairo-script -I. -I../../../../gtk/source/cairo-1.17.8 -Isrc -I../../../../gtk/source/cairo-1.17.8/src -I/Users/ben/gtk/inst/include/libpng16 -I/Users/ben/gtk/inst/include/freetype2 -I/Users/ben/gtk/inst/include/harfbuzz -I/Users/ben/gtk/inst/include/glib-2.0 -I/Users/ben/gtk/inst/lib/glib-2.0/include -I/opt/homebrew/Cellar/brotli/1.1.0/include -I/Users/ben/gtk/inst/include/pixman-1 -I/opt/homebrew/Cellar/lzo/2.10/include/lzo -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/Users/ben/gtk/inst/include -fcolor-diagnostics -Wall -Winvalid-pch -Wextra -O0 -g -Wmissing-declarations -Werror-implicit-function-declaration -Wpointer-arith -Wwrite-strings -Wsign-compare -Wpacked -Wswitch-enum -Wmissing-format-attribute -Wvolatile-register-var -Wstrict-aliasing=2 -Winit-self -Wno-missing-field-initializers -Wno-unused-parameter -Wno-attributes -Wno-long-long -Winline -Wno-unused-but-set-variable -Wno-enum-conversion -fno-strict-aliasing -fno-common -D_GNU_SOURCE -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -mmacosx-version-min=12 -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -MD -MQ util/cairo-script/libcairo-script-interpreter.2.dylib.p/cairo-script-operators.c.o -MF util/cairo-script/libcairo-script-interpreter.2.dylib.p/cairo-script-operators.c.o.d -o util/cairo-script/libcairo-script-interpreter.2.dylib.p/cairo-script-operators.c.o -c ../../../../gtk/source/cairo-1.17.8/util/cairo-script/cairo-script-operators.c
../../../../gtk/source/cairo-1.17.8/util/cairo-script/cairo-script-operators.c:63:10: fatal error: 'lzo/lzo2a.h' file not found

I sollved this by adding 'lzo' as a new module (not 100% sure this is completely right, but it seems to work):

a/modulesets-stable/gtk-osx.modules b/modulesets-stable/gtk-osx.modules
index 276bd7b..dae834d 100644
--- a/modulesets-stable/gtk-osx.modules
+++ b/modulesets-stable/gtk-osx.modules
@@ -39,6 +39,9 @@
   <repository name="sourceforge"
               href="http://downloads.sourceforge.net/sourceforge/"
               type="tarball" />
+  <repository name="oberhummer"
+              href="https://www.oberhumer.com/opensource/lzo/download/"
+              type="tarball" />
   <repository name="system"
               type="system" />
   <!--
@@ -194,7 +197,7 @@
   <!--
     disable-arm-a64-neon and disable-arm-neon are reuquired when building on
     Apple Silicon. See https://gitlab.freedesktop.org/pixman/pixman/-/issues/59
-  -->
+  -->
   <autotools id="pixman"
              autogen-sh="configure"
              autogenargs="--disable-gtk --disable-mmx --disable-arm-a64-neon --disable-arm-neon">
@@ -206,6 +209,15 @@
       <dep package="meta-gtk-osx-bootstrap" />
     </after>
   </autotools>
+  <!---->
+  <autotools id="lzo"
+             autogen-sh="configure"
+             autogenargs="--disable-dependency-tracking">
+    <branch module="lzo-2.10.tar.gz"
+            version="2.10"
+            hash="sha1:4924676a9bae5db58ef129dc1cebce3baa3c4b5d"
+            repo="oberhummer" />
+  </autotools>
   <!--
     cairo doesn't really need fontconfig, but if Pango finds
     freetype it insists that it has to have fontconfig too and that
@@ -222,6 +234,7 @@
       <dep package="meta-gtk-osx-bootstrap" />
       <dep package="harfbuzz-no-cairo" />
       <dep package="fontconfig" />
+      <dep package="lzo" />
     </dependencies>
   </meson>
   <!---->

but, that moves on to the next error!

FAILED: gtk/gtkresources.c
/Users/ben/gtk/inst/bin/glib-compile-resources gtk/gtk.gresources.xml --sourcedir /Users/ben/.cache/jhbuild/build/gtk+-3.24.38/gtk --sourcedir /Users/ben/gtk/source/gtk+-3.24.38/gtk --sourcedir ../../../../gtk/source/gtk+-3.24.38/gtk --c-name _gtk --internal --generate --target gtk/gtkresources.c --manual-register --dependency-file gtk/gtkresources.c.d
dyld[35156]: Symbol not found: _jpeg_CreateCompress
  Referenced from: <8899F39C-F1B8-3AC7-8FAC-BFDD3BA67DE4> /Users/ben/gtk/inst/lib/libgdk_pixbuf-2.0.0.dylib
  Expected in:     <no uuid> unknown
gtk/gtk.gresources.xml: Child process killed by signal 6.
[18/1529] Generating demos/gtk-demo/gtkdemo_resources_h with a custom command
FAILED: demos/gtk-demo/gtkdemo_resources.h
/Users/ben/gtk/inst/bin/glib-compile-resources ../../../../gtk/source/gtk+-3.24.38/demos/gtk-demo/demo.gresource.xml --sourcedir ../../../../gtk/source/gtk+-3.24.38/demos/gtk-demo/. --sourcedir ../../../../gtk/source/gtk+-3.24.38/demos/gtk-demo --internal --generate --target demos/gtk-demo/gtkdemo_resources.h
dyld[35180]: Symbol not found: _jpeg_CreateCompress
  Referenced from: <8899F39C-F1B8-3AC7-8FAC-BFDD3BA67DE4> /Users/ben/gtk/inst/lib/libgdk_pixbuf-2.0.0.dylib
  Expected in:     <no uuid> unknown
../../../../gtk/source/gtk+-3.24.38/demos/gtk-demo/demo.gresource.xml: Child process killed by signal 6.
[19/1529] Generating gtk/gtkresources_h with a custom command
FAILED: gtk/gtkresources.h
/Users/ben/gtk/inst/bin/glib-compile-resources gtk/gtk.gresources.xml --sourcedir /Users/ben/.cache/jhbuild/build/gtk+-3.24.38/gtk --sourcedir /Users/ben/gtk/source/gtk+-3.24.38/gtk --sourcedir ../../../../gtk/source/gtk+-3.24.38/gtk --c-name _gtk --internal --generate --target gtk/gtkresources.h --manual-register
dyld[35155]: Symbol not found: _jpeg_CreateCompress
  Referenced from: <8899F39C-F1B8-3AC7-8FAC-BFDD3BA67DE4> /Users/ben/gtk/inst/lib/libgdk_pixbuf-2.0.0.dylib
  Expected in:     <no uuid> unknown
gtk/gtk.gresources.xml: Child process killed by signal 6.
[20/1529] Generating demos/gtk-demo/gtkdemo_resources_c with a custom command
FAILED: demos/gtk-demo/gtkdemo_resources.c
/Users/ben/gtk/inst/bin/glib-compile-resources ../../../../gtk/source/gtk+-3.24.38/demos/gtk-demo/demo.gresource.xml --sourcedir ../../../../gtk/source/gtk+-3.24.38/demos/gtk-demo/. --sourcedir ../../../../gtk/source/gtk+-3.24.38/demos/gtk-demo --internal --generate --target demos/gtk-demo/gtkdemo_resources.c --dependency-file demos/gtk-demo/gtkdemo_resources.c.d
dyld[35179]: Symbol not found: _jpeg_CreateCompress
  Referenced from: <8899F39C-F1B8-3AC7-8FAC-BFDD3BA67DE4> /Users/ben/gtk/inst/lib/libgdk_pixbuf-2.0.0.dylib
  Expected in:     <no uuid> unknown
../../../../gtk/source/gtk+-3.24.38/demos/gtk-demo/demo.gresource.xml: Child process killed by signal 6.
[28/1529] Generating gtk/gtktypebuiltins.h with a custom command (wrapped by meson to capture output)
ninja: build stopped: subcommand failed.
*** Error during phase build of gtk+-3.0: ########## Error running ninja   *** [31/138]

totaam commented 1 year ago

OK, now I am getting build failure with cairo, which is looking for a LZO header.

Oh odd. I have never seen this one!

dyld[35179]: Symbol not found: _jpeg_CreateCompress

Ah, this one may well be: https://github.com/Xpra-org/gtk-osx-build/issues/19#issuecomment-878970323 Just install_name_tool then force rebuild of anything linked with libjpeg.

puremourning commented 1 year ago

Very strange.

ben@Bens-MacBook-Pro gdk-pixbuf-2.42.10 % otool -L /Users/ben/gtk/inst/lib/libgdk_pixbuf-2.0.0.dylib
/Users/ben/gtk/inst/lib/libgdk_pixbuf-2.0.0.dylib:
    /Users/ben/gtk/inst/lib/libgdk_pixbuf-2.0.0.dylib (compatibility version 4201.0.0, current version 4201.10.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.0.0)
    /Users/ben/gtk/inst/lib/libglib-2.0.0.dylib (compatibility version 7601.0.0, current version 7601.3.0)
    /Users/ben/gtk/inst/lib/libintl.8.dylib (compatibility version 12.0.0, current version 12.0.0)
    /Users/ben/gtk/inst/lib/libgobject-2.0.0.dylib (compatibility version 7601.0.0, current version 7601.3.0)
    /Users/ben/gtk/inst/lib/libgmodule-2.0.0.dylib (compatibility version 7601.0.0, current version 7601.3.0)
    /Users/ben/gtk/inst/lib/libgio-2.0.0.dylib (compatibility version 7601.0.0, current version 7601.3.0)
    /Users/ben/gtk/inst/lib/libpng16.16.dylib (compatibility version 57.0.0, current version 57.0.0)
    @rpath/libjpeg.62.dylib (compatibility version 62.0.0, current version 62.3.0)
    /Users/ben/gtk/inst/lib/libtiff.6.dylib (compatibility version 7.0.0, current version 7.2.0)

ben@Bens-MacBook-Pro gdk-pixbuf-2.42.10 % nm ~/gtk/inst/lib/libjpeg.62.dylib | grep Create
000000000000d628 T _jpeg_CreateCompress
000000000002e808 T _jpeg_CreateDecompress

totaam commented 1 year ago

@rpath/libjpeg.62.dylib

That's the broken bit that needs install_name_tool then a rebuild of gdk-pixbuf.

puremourning commented 1 year ago

Yes, that did work. For the record I did this:

jhbuild shell
cd $JHBUILD_PREFIX/lib
install_name_tool -id $PWD/libjpeg.62.dylib libjpeg.62.3.0.dylib
install_name_tool -change @rpath/libjpeg.62.dylib $PWD/libjpeg.62.dylib libtiff.6.dylib
jhbuild buildone -f gdk-pixbuf
jhbuild buildone -f cairo
jhbuild build

I'll look to see if I can encourage cmake to actually do that, as the -DCMAKE_MACOSX_RPATH=TRUE/FALSE doesn't appear to do anything.

Anyway. Next problem, need to cpan install XML::Simple, again probably due to homebrew perl.

And on to the next one:

*** Configuring xxhash *** [46/138]
cmake -DCMAKE_INSTALL_PREFIX=/Users/ben/gtk/inst -DCMAKE_INSTALL_LIBDIR=lib -G Ninja -DCMAKE_INSTALL_PREFIX=  /Users/ben/gtk/source/xxHash-0.8.2
CMake Warning:
  Ignoring extra path from command line:

   "/Users/ben/gtk/source/xxHash-0.8.2"

CMake Error: The source directory "/Users/ben/gtk/source/xxHash-0.8.2" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
*** Error during phase configure of xxhash: ########## Error running cmake -DCMAKE_INSTALL_PREFIX=/Users/ben/gtk/inst -DCMAKE_INSTALL_LIBDIR=lib -G Ninja -DCMAKE_INSTALL_PREFIX=  /Users/ben/gtk/source/xxHash-0.8.2 *** [46/138]

"Fixed" that:

index 2987b84..26321c9 100644
--- a/xpra-tools.modules
+++ b/xpra-tools.modules
@@ -96,7 +96,7 @@
                        repo="github.com"/>
        </cmake>

-       <cmake id="xxhash" cmakeargs="-DCMAKE_INSTALL_PREFIX=" autogen-sh="configure">
+       <cmake id="xxhash" cmakeargs="-DCMAKE_MACOSX_RPATH=FALSE" autogen-sh="configure" cmakedir="cmake_unofficial">
                <branch module="/Cyan4973/xxHash/archive/refs/tags/v0.8.2.tar.gz"
                        version="0.8.2" checkoutdir="xxHash-0.8.2"
                        hash="sha256:baee0c6afd4f03165de7a4e67988d16f0f2b257b51d0e3cb91909302a26a79c4"

And on to the next error...

*** Building python3-bcrypt *** [68/138]
python3 setup.py build --build-base /Users/ben/.cache/jhbuild/build/bcrypt-4.0.1

        =============================DEBUG ASSISTANCE==========================
        If you are seeing an error here please try the following to
        successfully install bcrypt:

        Upgrade to the latest pip and try again. This will fix errors for most
        users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
        =============================DEBUG ASSISTANCE==========================

Traceback (most recent call last):
  File "/Users/ben/gtk/source/bcrypt-4.0.1/setup.py", line 11, in <module>
    from setuptools_rust import RustExtension
  File "/Users/ben/gtk/inst/lib/python3.11/site-packages/setuptools_rust/__init__.py", line 1, in <module>
    from .build import build_rust
  File "/Users/ben/gtk/inst/lib/python3.11/site-packages/setuptools_rust/build.py", line 27, in <module>
    from typing_extensions import Literal
ModuleNotFoundError: No module named 'typing_extensions'

EDIT: Just realised this is documented in the build instructions. Will do that.!

totaam commented 1 year ago

I'll look to see if I can encourage cmake to actually do that, as the -DCMAKE_MACOSX_RPATH=TRUE/FALSE doesn't appear to do anything.

Good, so it's not just me. I had tried to fix it and then gave up on it.

Error during phase configure of xxhash

Thanks! Applied.

puremourning commented 1 year ago

For the record. Next problem : libvorbis doesn't build due to an unsupported linker flag.

Homebrew fixes this with a patch: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/lib/libvorbis.rb#L45C12-L45C12

I've manually edited configure for now, but will create a patch file to fix it later.

puremourning commented 1 year ago

Next, I had to remove --enable-runtime-cpu-detection for libvpx as that's not supported on this CPU. Ideally this would be switched in the config based on architecture.

Then, I changed the xar module to use the subdirectory as this was also failing:

    <autotools id="xar" autogen-sh="autogen.sh" supports-non-srcdir-builds="no">
        <branch repo="github" module="typeling1578/xar" subdir="xar"/>
    </autotools>

AND...

*** success *** [138/138]

\o/

Now to build Xpra!

ben@Bens-MacBook-Pro MacOS % ./make-app.sh
Building Xpra for Python 3.11
the xpra html5 client must be installed in /Users/ben/gtk/inst/share/xpra/www/

OK more homework for me!

CLIENT_ONLY=1 ./make-app.sh

Gets further and fails on py2app missing. I'll keep at it. So close now.

totaam commented 1 year ago

Then, I changed the xar module to use the subdirectory as this was also failing:

Thanks! Applied in https://github.com/Xpra-org/gtk-osx-build/commit/53da4eabcb0273951998280aebb35948d0f90288

*** success *** [138/138]

Nice!

CLIENT_ONLY=1 ./make-app.sh

There is a problem with CLIENT_ONLY=1 builds - I can't remember what it is, which is why there are none in the downloads folder. Try without?

puremourning commented 1 year ago

Without CLIENT_ONLY, I get 'the Xpra html5 client must be installed in ...' which I don't yet know how to solve.

With CLIENT_ONLY, I get this:

*******************************************************************************
py2app step:
./setup.py py2app  --without-server --without-shadow --without-proxy
 (see /Users/ben/repos/github/xpra/xpra/py2app.log for details - this may take a minute or two)
ERROR: py2app failed

* spng_encoder         : Y
* ssh                  : Y
* strict               : N
* tests                : N
* uinput               : N
* v4l2                 : N
* verbose              : N
* vpx                  : Y
* vsock                : N
* warn                 : Y
* webcam               : N
* webp                 : Y
* x11                  : N
* xdg_open             : N
* xinput               : N
Traceback (most recent call last):
  File "/Users/ben/repos/github/xpra/xpra/./setup.py", line 1938, in <module>
    import py2app    #@UnresolvedImport
    ^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'py2app'

is that what you were remembering?

totaam commented 1 year ago

Just install the xpra-html5 client, that's really easy.

puremourning commented 1 year ago

I did that and get the same error:

ben@Bens-MacBook-Pro MacOS % ./make-app.sh 
Building Xpra for Python 3.11
*******************************************************************************
Deleting existing xpra modules and temporary directories

*******************************************************************************
Building and installing locally
~/repos/github/xpra/xpra ~/repos/github/xpra/xpra/packaging/MacOS
updated ./xpra/src_info.py with:
* BRANCH               = master
* COMMIT               = gda232585f
* LOCAL_MODIFICATIONS  = 0
* REVISION             = 34474
'nvcc --version' failed with return code 127
stderr: None
updated ./xpra/build_info.py with:
* BUILD_BIT            = 64bit
* BUILD_CPU            = arm
* BUILD_DATE           = 2023-10-04
* BUILD_MACHINE        = arm64
* BUILD_OS             = MacOS 13.6
* BUILD_TIME           = 16:55
* BUILT_BY             = ben
* BUILT_ON             = Bens-MacBook-Pro.local
* COMPILER_VERSION     = Apple clang version 15.0.0 (clang-1500.0.40.1)
* CYTHON_VERSION       = 3.0.2
* LINKER_VERSION       = LLVM version 15.0.0 (static support for 29, runtime is 29)
* PYTHON_VERSION       = 3.11.5
* lib_gobject_introspection = 1.76.1
* lib_gtk              = 3.24.38
* lib_py3cairo         = 1.24.0
* lib_pygobject        = 3.44.1
* lib_python3          = 3.11
* lib_vpx              = 1.13.1
* lib_x11              = 1.8.6
* lib_x264             = 0.164.x
* lib_xext             = 1.3.5
./setup.py clean
Xpra version 6.0
found Cython version 3.0.2
found pkg-config version: 0.29.2
ARCH='arm64'
using INCLUDE_DIRS=['/Users/ben/repos/github/xpra/devroot/.new_local/share/virtualenvs/etc-zZzKIFJg/include']
CC='/Applications/Xcode.app/Contents/Developer/usr/bin/gcc'
CPP='cpp'
found 10 logical CPUs
./setup.py build_ext  -j 10
 (see /Users/ben/repos/github/xpra/xpra/build_ext.log for details - this may take a minute or two)
./setup.py install 
 (see /Users/ben/repos/github/xpra/xpra/install.log for details)
OK

*******************************************************************************
py2app step:
./setup.py py2app 
 (see /Users/ben/repos/github/xpra/xpra/py2app.log for details - this may take a minute or two)
ERROR: py2app failed

* spng_encoder         : Y
* ssh                  : Y
* strict               : N
* tests                : N
* uinput               : N
* v4l2                 : N
* verbose              : N
* vpx                  : Y
* vsock                : N
* warn                 : Y
* webcam               : N
* webp                 : Y
* x11                  : N
* xdg_open             : N
* xinput               : N
Traceback (most recent call last):
  File "/Users/ben/repos/github/xpra/xpra/./setup.py", line 1938, in <module>
    import py2app    #@UnresolvedImport
    ^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'py2app'

puremourning commented 1 year ago

py2app.log

puremourning commented 1 year ago

OK I fixed that:

ben@Bens-MacBook-Pro MacOS % git diff
diff --git a/packaging/MacOS/make-app.sh b/packaging/MacOS/make-app.sh
index 171853b71..f212759b9 100755
--- a/packaging/MacOS/make-app.sh
+++ b/packaging/MacOS/make-app.sh
@@ -87,7 +87,7 @@ if [ "$?" != "0" ]; then
        exit 1
 fi
 #get the version and build info from the python build records:
-export PYTHONPATH="."
+export PYTHONPATH="$(pwd)${PYTHONPATH:+:$PYTHONPATH}"
 VERSION=`${PYTHON} -c "from xpra import __version__;import sys;sys.stdout.write(__version__)"`
 REVISION=`${PYTHON} -c "from xpra import src_info;import sys;sys.stdout.write(str(src_info.REVISION))"`
 REV_MOD=`${PYTHON} -c "from xpra import src_info;import sys;sys.stdout.write(['','M'][src_info.LOCAL_MODIFICATIONS>0])"`
ben@Bens-MacBook-Pro MacOS %

now gtk-mac-bundler failed. But more progress!

puremourning commented 1 year ago

it was my environment messing up the paths. I sorted this out so that ghbuild shell is cleaner and... it's built!

\o/

And it works!

Or, well it looked like it...

But it's not crashing with undefined symbols. Nearly there.

puremourning commented 1 year ago

OK this is why it failed:

2023-10-05 10:41:07,441 Warning: server connection failure:
2023-10-05 10:41:07,442  Connection failed: disconnected before the session could be established
2023-10-05 10:41:07,442  connection error
2023-10-05 10:41:07,442  client failed to specify any supported encodings

hmm...

ben@Bens-MacBook-Pro MacOS % ./image/Xpra.app/Contents/MacOS/Xpra  encoding
grep: /usr/share/locale/locale.alias: No such file or directory
<frozen importlib._bootstrap>:673: ImportWarning: DynamicImporter.exec_module() not found; falling back to load_module()
modules found:
* csc_cython           : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/csc_cython/converter.cpython-311-darwin.so
* csc_libyuv           : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/libyuv/converter.cpython-311-darwin.so
* dec_gstreamer        : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/gstreamer/decoder.py
* dec_jpeg             : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/jpeg/decoder.cpython-311-darwin.so
* dec_openh264         : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/openh264/decoder.cpython-311-darwin.so
* dec_pillow           : dlopen(/Users/ben/repos/github/xpra/xpra/packaging/MacOS/image/Xpra.app/Contents/Resources/lib/python/lib-dynload/PIL/_imaging.so, 0x0002): Library not loaded: @executable_path/../Frameworks/libxcb.1.1.0.dylib
  Referenced from: <7851DF75-48C2-30F6-AC9D-D6BF464E5602> /Users/ben/repos/github/xpra/xpra/packaging/MacOS/image/Xpra.app/Contents/Resources/lib/python3.11/lib-dynload/PIL/_imaging.so
  Reason: tried: '/Users/ben/repos/github/xpra/xpra/packaging/MacOS/image/Xpra.app/Contents/Resources/lib/libxcb.1.1.0.dylib' (no such file), '/Users/ben/repos/github/xpra/xpra/packaging/MacOS/image/Xpra.app/Contents/Resources/Frameworks/libxcb.1.1.0.dylib' (no such file)
* dec_spng             : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/spng/decoder.cpython-311-darwin.so
* dec_vpx              : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/vpx/decoder.cpython-311-darwin.so
* dec_webp             : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/webp/decoder.cpython-311-darwin.so
* enc_gstreamer        : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/gstreamer/encoder.py
* enc_jpeg             : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/jpeg/encoder.cpython-311-darwin.so
* enc_openh264         : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/openh264/encoder.cpython-311-darwin.so
* enc_pillow           : dlopen(/Users/ben/repos/github/xpra/xpra/packaging/MacOS/image/Xpra.app/Contents/Resources/lib/python/lib-dynload/PIL/_imaging.so, 0x0002): Library not loaded: @executable_path/../Frameworks/libxcb.1.1.0.dylib
  Referenced from: <7851DF75-48C2-30F6-AC9D-D6BF464E5602> /Users/ben/repos/github/xpra/xpra/packaging/MacOS/image/Xpra.app/Contents/Resources/lib/python3.11/lib-dynload/PIL/_imaging.so
  Reason: tried: '/Users/ben/repos/github/xpra/xpra/packaging/MacOS/image/Xpra.app/Contents/Resources/lib/libxcb.1.1.0.dylib' (no such file), '/Users/ben/repos/github/xpra/xpra/packaging/MacOS/image/Xpra.app/Contents/Resources/Frameworks/libxcb.1.1.0.dylib' (no such file)
* enc_rgb              : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/argb/encoder.py
* enc_spng             : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/spng/encoder.cpython-311-darwin.so
* enc_vpx              : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/vpx/encoder.cpython-311-darwin.so
* enc_webp             : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/webp/encoder.cpython-311-darwin.so
* enc_x264             : image/Xpra.app/Contents/Resources/lib/python/xpra/codecs/x264/encoder.cpython-311-darwin.so

codecs versions:
* cython                          : 4.2
* gstreamer                       : 5.0
* jpeg                            : 1.0
* libyuv                          : 1.0
* openh264                        : 2.3.1
* rgb                             : 4.3
* spng                            : 0.7.4
* vpx                             : 1.13.1
* webp                            : 1.3.2
* x264                            : 164
ben@Bens-MacBook-Pro MacOS %

I'll have to dig into it another time. Thanks for the help so far though!

totaam commented 1 year ago

Library not loaded: @executable_path/../Frameworks/libxcb.1.1.0.dylib

That's most definitely wrong. Nothing should be linked with X11 libraries. And running without the pillow codecs is not supported - might work, might not.

I'll have to dig into it another time

Could be a compatibility error. You're building git master? What version is the server?

puremourning commented 1 year ago

It was indeed a compatibility error. Server was using v. old version. Upgraded to latest 5.x release and my Xpra client connects!

\o/

Going to have to find some solid time to polish up all of the changes to the build. But I guess we're not really closer to understanding why we are getting crashes in the released version and if we can mitigate that.

First though I need to work out why this is happening (DPI issue?)

totaam commented 1 year ago

First though I need to work out why this is happening (DPI issue?)

What am I looking at? Is the xterm meant to cover the whole window? If so, try force enabling opengl with --opengl=force. IIRC, MacOS displays lie about their resolution and expect us to upscale things.

puremourning commented 1 year ago

First though I need to work out why this is happening (DPI issue?)

What am I looking at? Is the xterm meant to cover the whole window?

Yes. I started the client and ran "xterm" from the server. Client is in "seamless" mode.

If so, try force enabling opengl with --opengl=force.

Actually, --opengl=no resolves it.

Here's the logs with --opengl=no

ben@Bens-MacBook-Pro Xpra % ~/Desktop/Xpra.app/Contents/MacOS/Xpra attach --opengl=no ssh://stol-core1/1001
grep: /usr/share/locale/locale.alias: No such file or directory
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:635: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'B@:#B')
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:641: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'v@:I')
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:664: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'v@:I')
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:669: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'v@:I')

(Xpra:20258): Gtk-CRITICAL **: 18:32:22.977: gtk_window_add_accel_group: assertion 'GTK_IS_WINDOW (window)' failed
grep: /usr/share/locale/locale.alias: No such file or directory
2023-10-09 18:32:23,092 Xpra GTK3 client version 6.0-r34474M (gda232585f) beta
2023-10-09 18:32:23,105  running on Mac OS X
2023-10-09 18:32:23,105  cpython 3.11
2023-10-09 18:32:23,112 window icon overlay requires python-pillow
grep: /usr/share/locale/locale.alias: No such file or directory
<frozen importlib._bootstrap>:673: ImportWarning: DynamicImporter.exec_module() not found; falling back to load_module()
<frozen importlib._bootstrap>:673: ImportWarning: DynamicImporter.exec_module() not found; falling back to load_module()
2023-10-09 18:32:23,392 GStreamer version 1.22.6
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:635: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'B@:#B')
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:641: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'v@:I')
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:664: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'v@:I')
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:669: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'v@:I')
2023-10-09 18:32:23,441 created unix domain socket '/Users/ben/.xpra/clients/Bens-MacBook-Pro.local-20258'
<frozen importlib._bootstrap>:673: ImportWarning: DynamicImporter.exec_module() not found; falling back to load_module()

(Xpra:20258): Gtk-CRITICAL **: 18:32:23.490: gtk_window_add_accel_group: assertion 'GTK_IS_WINDOW (window)' failed
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/tray.py:109: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  self.disconnect_dock_item = Gtk.MenuItem("Disconnect")
2023-10-09 18:32:23,614 Connected (version 2.0, client OpenSSH_8.0)
2023-10-09 18:32:24,065 Authentication (publickey) successful!
2023-10-09 18:32:25,798 ssh server OS is 'linux-gnu'
2023-10-09 18:32:27,623 paramiko SSH agent forwarding enabled
2023-10-09 18:32:27,715  keyboard settings: layout=gb
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/gtk/gtk_util.py:236: DeprecationWarning: Gdk.Screen.get_width is deprecated
  sw, sh = screen.get_width(), screen.get_height()
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/gtk/gtk_util.py:236: DeprecationWarning: Gdk.Screen.get_height is deprecated
  sw, sh = screen.get_width(), screen.get_height()
2023-10-09 18:32:27,722  desktop size is 5120x1440:
2023-10-09 18:32:27,723   bens-macbook-pro.local (1355x381 mm - DPI: 96x96)
2023-10-09 18:32:27,723     1                2560x1440 at    0x0    (599x340 mm - DPI: 109x108) workarea: 2560x1415 at    0x25
2023-10-09 18:32:27,723     monitor 2        2560x1440 at 2560x0    (599x340 mm - DPI: 109x108) workarea: 2560x1330 at 2560x25
2023-10-09 18:32:29,067 enabled remote logging
2023-10-09 18:32:29,068 Xpra X11 seamless server version 5.0
2023-10-09 18:32:29,080 Attached to xpra server at ssh://benjamin.jackson@stol-core1.orcsoftware.com/1001
2023-10-09 18:32:29,080  (press Control-C to detach)

grep: /usr/share/locale/locale.alias: No such file or directory
2023-10-09 18:32:29,289 running, 1 windows
<frozen importlib._bootstrap>:673: ImportWarning: DynamicImporter.exec_module() not found; falling back to load_module()
<frozen importlib._bootstrap>:673: ImportWarning: DynamicImporter.exec_module() not found; falling back to load_module()
2023-10-09 18:32:30,020 audio playback using 'flac' audio codec

And with --opengl=force:

ben@Bens-MacBook-Pro Xpra % ~/Desktop/Xpra.app/Contents/MacOS/Xpra attach --opengl=force ssh://stol-core1/1001
grep: /usr/share/locale/locale.alias: No such file or directory
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:635: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'B@:#B')
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:641: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'v@:I')
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:664: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'v@:I')
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:669: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'v@:I')

(Xpra:20395): Gtk-CRITICAL **: 18:33:07.989: gtk_window_add_accel_group: assertion 'GTK_IS_WINDOW (window)' failed
grep: /usr/share/locale/locale.alias: No such file or directory
2023-10-09 18:33:08,096 Xpra GTK3 client version 6.0-r34474M (gda232585f) beta
2023-10-09 18:33:08,109  running on Mac OS X
2023-10-09 18:33:08,109  cpython 3.11
2023-10-09 18:33:08,116 window icon overlay requires python-pillow
grep: /usr/share/locale/locale.alias: No such file or directory
<frozen importlib._bootstrap>:673: ImportWarning: DynamicImporter.exec_module() not found; falling back to load_module()
<frozen importlib._bootstrap>:673: ImportWarning: DynamicImporter.exec_module() not found; falling back to load_module()
2023-10-09 18:33:08,398 GStreamer version 1.22.6
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:635: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'B@:#B')
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:641: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'v@:I')
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:664: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'v@:I')
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gui.py:669: DeprecationWarning: Usage objc.typedSelector instead of objc.signature
  @objc.signature(b'v@:I')
2023-10-09 18:33:08,448 created unix domain socket '/Users/ben/.xpra/clients/Bens-MacBook-Pro.local-20395'
<frozen importlib._bootstrap>:673: ImportWarning: DynamicImporter.exec_module() not found; falling back to load_module()
2023-10-09 18:33:08,757 OpenGL enabled on 'Apple M1 Max'

(Xpra:20395): Gtk-CRITICAL **: 18:33:08.765: gtk_window_add_accel_group: assertion 'GTK_IS_WINDOW (window)' failed
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/tray.py:109: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  self.disconnect_dock_item = Gtk.MenuItem("Disconnect")
2023-10-09 18:33:08,885 Connected (version 2.0, client OpenSSH_8.0)
2023-10-09 18:33:09,351 Authentication (publickey) successful!
2023-10-09 18:33:11,153 ssh server OS is 'linux-gnu'
2023-10-09 18:33:12,999 paramiko SSH agent forwarding enabled
2023-10-09 18:33:13,094  keyboard settings: layout=gb
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/gtk/gtk_util.py:236: DeprecationWarning: Gdk.Screen.get_width is deprecated
  sw, sh = screen.get_width(), screen.get_height()
/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/gtk/gtk_util.py:236: DeprecationWarning: Gdk.Screen.get_height is deprecated
  sw, sh = screen.get_width(), screen.get_height()
2023-10-09 18:33:13,101  desktop size is 5120x1440:
2023-10-09 18:33:13,101   bens-macbook-pro.local (1355x381 mm - DPI: 96x96)
2023-10-09 18:33:13,101     1                2560x1440 at    0x0    (599x340 mm - DPI: 109x108) workarea: 2560x1415 at    0x25
2023-10-09 18:33:13,101     monitor 2        2560x1440 at 2560x0    (599x340 mm - DPI: 109x108) workarea: 2560x1330 at 2560x25
2023-10-09 18:33:14,444 enabled remote logging
2023-10-09 18:33:14,445 Xpra X11 seamless server version 5.0
2023-10-09 18:33:14,456 Attached to xpra server at ssh://benjamin.jackson@stol-core1.orcsoftware.com/1001
2023-10-09 18:33:14,456  (press Control-C to detach)

2023-10-09 18:33:14,510 running, 1 windows
grep: /usr/share/locale/locale.alias: No such file or directory
<frozen importlib._bootstrap>:673: ImportWarning: DynamicImporter.exec_module() not found; falling back to load_module()
<frozen importlib._bootstrap>:673: ImportWarning: DynamicImporter.exec_module() not found; falling back to load_module()
2023-10-09 18:33:15,410 audio playback using 'flac' audio codec

Attached with -d opengl in case useful

opengl.txt

IIRC, MacOS displays lie about their resolution and expect us to upscale things.

Yes, this is typical. There are APIs to get the actual bounds and "virtual" bounds IIRC.

totaam commented 1 year ago

DeprecationWarning: Usage objc.typedSelector instead of objc.signature

Moved to #4028

Actually, --opengl=no resolves it.

That's odd. xterm is text and should not be using GL: https://github.com/Xpra-org/xpra/blob/e09c432bd6e984055c64edeac6a43957f2c61e10/xpra/client/gtk3/client_base.py#L1367-L1368

There are APIs to get the actual bounds and "virtual" bounds IIRC.

Can you try to run https://github.com/Xpra-org/xpra/blob/master/xpra/platform/darwin/gui.py in debug mode and see if there is a backingScaleFactor: https://github.com/Xpra-org/xpra/blob/e09c432bd6e984055c64edeac6a43957f2c61e10/xpra/platform/darwin/gui.py#L262

the backingscalefactor documentation says we should use NSView instead, but I don't see how. If this scale factor is not 1, then we should be able to plug it into the opengl renderer fairly easily.

puremourning commented 1 year ago

backingScaleFactor is 2. Full log with -d osx attached:

ben@Bens-MacBook-Pro xpra % grep backingScale osx.log
2023-10-10 18:53:01,443  backingScaleFactor=2.0
2023-10-10 18:53:01,443  backingScaleFactor=2.0
2023-10-10 18:53:01,444  backingScaleFactor=2.0
2023-10-10 18:53:01,444  backingScaleFactor=2.0
2023-10-10 18:53:01,450  backingScaleFactor=2.0
2023-10-10 18:53:01,450  backingScaleFactor=2.0
2023-10-10 18:53:01,450  backingScaleFactor=2.0
2023-10-10 18:53:01,450  backingScaleFactor=2.0
2023-10-10 18:53:01,451  backingScaleFactor=2.0
2023-10-10 18:53:01,451  backingScaleFactor=2.0
2023-10-10 18:53:01,451  backingScaleFactor=2.0
2023-10-10 18:53:01,451  backingScaleFactor=2.0

osx.log

puremourning commented 1 year ago

we should use NSView instead

My recollection on this is hazy, but I vaguely recall that you're supposed to ask the view for its bounds or use use the various functions like convertRectToBacking etc. from here: https://developer.apple.com/documentation/appkit/nsview/view_coordinates

But yeah, multiplying by 1/NSScreen.backingScaleFactor makes things work more like other OSes with scaled UIs (i.e. using pixels rather than points)

puremourning commented 1 year ago

Perhaps this discussion is relevant too:

https://developer.apple.com/documentation/appkit/nsview/1483817-bounds#discussion

totaam commented 1 year ago

@puremourning the commit above should work and give you scaling with OpenGL enabled. Can you try it?

Notes: We already had window scaling logic so that the client can upscale / downscale (though not per-window yet: #3454). I couldn't hook into that because this use-case is slightly different and depends on the window context when running on multi-monitor setups: each "screen" may have a different backingScaleFactor. So instead, we query the backingScaleFactor just before painting the screen in present_fbo. I have no idea what happens if the window is straddling multiple screens.. We use GL_LINEAR upscaling, which may or may not be the right choice. Perhaps this should take the window's content type into account to choose a better filter? I don't have the hardware to test, but I have added a cheat environment variable to visually test the scaling on X11: XPRA_OPENGL_SCALE_FACTOR=0.666 xpra attach .. Perhaps this scaling factor should be forwarded to the server side so it can do something with it? We could conceivably run the server's virtual display at double the resolution and let the xterm adjust its font size automatically - this should work, but then we would have to fight with GTK for all screen relative coordinates (windows, pointer events, system tray, ... all of them)

puremourning commented 1 year ago

Thanks! Im away from my Mac today but will test drive tomorrow. My guess is that up scaling client side will lead to best overall experience and reduce bandwidth, but server side full res framebuffer will be best for fidelity. Probably both ultimately- Depends on how good scaling is in X and the various UI frameworks these days I suppose. Last I looked it wasn't great but that was a long while ago.

puremourning commented 1 year ago

Hmm, with latest master, make-app.sh is failing:

Traceback (most recent call last):
  File "/Users/ben/repos/github/xpra/devroot/.new_local/bin/gtk-mac-bundler", line 6, in <module>
    bundler.main.main(sys.argv[1:])
  File "/Users/ben/gtk/source/gtk-mac-bundler/bundler/main.py", line 19, in main
    bundler.run()
  File "/Users/ben/gtk/source/gtk-mac-bundler/bundler/bundler.py", line 406, in run
    self.copy_binaries()
  File "/Users/ben/gtk/source/gtk-mac-bundler/bundler/bundler.py", line 186, in copy_binaries
    if path.compute_destination(self.project) in binaries:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ben/gtk/source/gtk-mac-bundler/bundler/project.py", line 147, in compute_destination
    raise ValueError (f'Invalid path, missing or invalid dest {self.dest}')
ValueError: Invalid path, missing or invalid dest None
ERROR: gtk-mac-bundler failed

I'm going to wipe everything and start again, see if that helps.

puremourning commented 1 year ago

OK rebuilt it, but got this:

here's the trace with -d osx,opengl

2023-10-13 22:02:36,015  do_run_glcheck(..)
2023-10-13 22:02:36,015  Traceback (most recent call last):
2023-10-13 22:02:36,015    File "/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/scripts/main.py", line 2427, in do_run_glcheck
2023-10-13 22:02:36,015      opengl_props, gl_client_window_module = get_gl_client_window_module(force_enable)
2023-10-13 22:02:36,015                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-13 22:02:36,015    File "/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/client/gl/window.py", line 32, in get_gl_client_window_module
2023-10-13 22:02:36,015      opengl_props = native_window.check_support(force_enable)
2023-10-13 22:02:36,015                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-13 22:02:36,015    File "/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/client/gl/gtk3/native_window.py", line 21, in check_support
2023-10-13 22:02:36,015      return GLContext().check_support(force_enable)  #pylint: disable=not-callable
2023-10-13 22:02:36,015             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-13 22:02:36,015    File "/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gl_context.py", line 135, in check_support
2023-10-13 22:02:36,015      with self.get_paint_context(tmp):
2023-10-13 22:02:36,016           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-10-13 22:02:36,016    File "/Users/ben/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/platform/darwin/gl_context.py", line 169, in get_paint_context
2023-10-13 22:02:36,016      self.scale_factor = nsview.screen().backingScaleFactor()
2023-10-13 22:02:36,016                          ^^^^^^^^^^^^^
2023-10-13 22:02:36,016  AttributeError: 'GdkQuartzView' object has no attribute 'screen'
2023-10-13 22:02:36,016  error='GdkQuartzView' object has no attribute 'screen'
2023-10-13 22:02:36,016  AGLContext.destroy() window_context=AGLWindowContext(<NSOpenGLContext: 0x600001dc9320>, <GdkQuartzView: 0x10da7f140>)
2023-10-13 22:02:36,016
2023-10-13 22:02:36,016  (Xpra:64007): Gdk-WARNING **: 22:02:35.982: losing last reference to undestroyed window
2023-10-13 22:02:36,016  run_glcheck(..) props={'success': False, 'message': "'GdkQuartzView' object has no attribute 'screen'"}
2023-10-13 22:02:36,017 OpenGL probe command returned 0 for command=['/Users/ben/Desktop/Xpra.app/Contents/Helpers/Xpra', 'opengl', '-d', 'opengl']
2023-10-13 22:02:36,017 probe took 658ms
2023-10-13 22:02:36,017 parsed OpenGL properties={'message': "'GdkQuartzView' object has no attribute 'screen'", 'success': 'False'}

totaam commented 1 year ago

@puremourning sorry about that, I can't test it here without the hardware. My VMs don't have OpenGL on MacOS.

The commit above moves things to the existing objective-c glue code, so at least it will be checked more thoroughly at compile time - and it does compile FWIW. So it should work, this time?

puremourning commented 1 year ago

No worries. I'll try it out! Appreciate the efforts.

totaam commented 10 months ago

libffi errors were meant to be fixed in https://github.com/Xpra-org/gtk-osx-build/issues/27

mdrolet01 commented 9 months ago

@puremourning I'm having a similar issue. I was wondering if you had any of the following issues? I can't get the pygobject-3.0.pc and py3cairo.pc linked up, as they appear to be missing, I.e:

./setup.py build_ext  --without-docs -j 14
 (see /Users/me/xpra/build_ext.log for details - this may take a minute or two)
~/xpra/packaging/MacOS
ERROR: build_ext failed

* tests                : No
* uinput               : No
* v4l2                 : No
* verbose              : No
* vpx                  : No
* vsock                : No
* warn                 : Yes
* webcam               : No
* webp                 : No
* x11                  : No
* xdg_open             : No
* xinput               : No
Warning: no x264 and no vpx support!
 you should enable at least one of these two video encodings
found Apple clang version: 15.0.0
ERROR: call to 'pkg-config --libs gtk+-3.0 pygobject-3.0' failed (err=Package pygobject-3.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `pygobject-3.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'pygobject-3.0' found

Also, did you end up getting this working smoothly for the new macs? Any thoughts here would be greatly appreciated.

totaam commented 9 months ago

@mdrolet01 have you followed the build instructions correctly? You seem to be missing a lot of the packages that would have been installed with the xpra moduleset - including pygobject-3.0.

mdrolet01 commented 9 months ago

I'm not sure I am.. I am following this: https://github.com/Xpra-org/xpra/blob/master/docs/Build/MacOS.md

After cloning this: https://github.com/Xpra-org/gtk-osx-build

totaam commented 9 months ago

@mdrolet01 that's right. My guess is that you're not building xpra's moduleset, perhaps the jhbuild config file is missing or misnamed. You can find a semi scripted version here: https://github.com/cpatulea/gtk-osx-build/actions/runs/7394059836/workflow

mdrolet01 commented 9 months ago

Thanks for the response. I tried to reproduce the workflow, but unfortunately I keep getting a similar error during the build of many of the (out of 16) components:

configure: error: in `/Users/me/.cache/jhbuild/build/flex-2.6.4': configure: error: C preprocessor "/lib/cpp" fails sanity check

I think I will wait until a working .dmg comes out, as building from source has taken a bit too much time for me. Thanks again though.

totaam commented 9 months ago

@mdrolet01 I believe you just need to change the key setup line in the jhbuild config file from x86_64 to arm64. Trying to build x86_64 on arm64 will error out that way pretty quickly.

cpatulea commented 9 months ago

@rpath/libjpeg.62.dylib

I think this may be due to libjpeg-turbo explicitly setting some target property MACOSX_RPATH: https://github.com/libjpeg-turbo/libjpeg-turbo/blob/be96fa0a403e5d084719757f7b8241d20393246d/sharedlib/CMakeLists.txt#L49

I disabled that line, then set -DCMAKE_MACOSX_RPATH=false, and finally I can get a libjpeg.62.dylib without "@rpath/...".

I still don't fully understand how to properly get the full path (like other libs) but I'm getting a bit further.

puremourning commented 9 months ago

FWIW when I was building this stuff successfully there were about 7 or 8 manual fix ups required including faffing rpath and other stuff. I thought I had written them all above in this thread.

I also have a bunch of changes to the build system which maybe I could share.

In the end I had to abandon using Xpra on my MacBook for work-related reasons so I never got round to polishing the changes. Unfortunately my brain has paged out all the knowledge about it now!

totaam commented 9 months ago

The nasm issue is fixed here: https://github.com/Xpra-org/gtk-osx-build/commit/d844b81d777e60a9b403f2ae853a4a33b9647ccc

@cpatulea our rpath issues are discussed in great detail by a meson developer here: https://github.com/Xpra-org/xpra/issues/4085#issuecomment-1876041460

gcardamone commented 9 months ago

I tried to fix pixman by upgrading to 0.42.2, but it looks like I get the same errors.

/Users/cardamone/gtk/source/pixman-0.42.2/pixman/pixman-arma64-neon-asm.S:3702:1: note: while in macro instantiation
generate_bilinear_scanline_func pixman_scaled_bilinear_scanline_0565_0565_SRC_asm_neon, 0565, 0565, 1, 1, 28, BILINEAR_FLAG_UNROLL_4
^
<instantiation>:13:20: error: unknown token in expression
    bilinear_store_&dst_fmt 1, v3, v4
                   ^
<instantiation>:125:5: note: while in macro instantiation
    bilinear_interpolate_last_pixel src_fmt, dst_fmt
    ^
/Users/cardamone/gtk/source/pixman-0.42.2/pixman/pixman-arma64-neon-asm.S:3702:1: note: while in macro instantiation
generate_bilinear_scanline_func pixman_scaled_bilinear_scanline_0565_0565_SRC_asm_neon, 0565, 0565, 1, 1, 28, BILINEAR_FLAG_UNROLL_4
^
<instantiation>:13:20: error: invalid operand
    bilinear_store_&dst_fmt 1, v3, v4
                   ^
<instantiation>:125:5: note: while in macro instantiation
    bilinear_interpolate_last_pixel src_fmt, dst_fmt
    ^
/Users/cardamone/gtk/source/pixman-0.42.2/pixman/pixman-arma64-neon-asm.S:3702:1: note: while in macro instantiation
generate_bilinear_scanline_func pixman_scaled_bilinear_scanline_0565_0565_SRC_asm_neon, 0565, 0565, 1, 1, 28, BILINEAR_FLAG_UNROLL_4
^
<instantiation>:150:1: error: unknown directive
.endfunc
^
/Users/cardamone/gtk/source/pixman-0.42.2/pixman/pixman-arma64-neon-asm.S:3702:1: note: while in macro instantiation
generate_bilinear_scanline_func pixman_scaled_bilinear_scanline_0565_0565_SRC_asm_neon, 0565, 0565, 1, 1, 28, BILINEAR_FLAG_UNROLL_4
^
make[2]: *** [Makefile:907: pixman-arma64-neon-asm.lo] Error 1
make[2]: Leaving directory '/Users/cardamone/.cache/jhbuild/build/pixman-0.42.2/pixman'
make[1]: *** [Makefile:514: all-recursive] Error 1
make[1]: Leaving directory '/Users/cardamone/.cache/jhbuild/build/pixman-0.42.2'
make: *** [Makefile:421: all] Error 2
*** Error during phase build of pixman: ########## Error running make -j 11  *** [15/137]

I'm going to add the workaround.

gcardamone commented 9 months ago

The nasm issue is fixed here: Xpra-org/gtk-osx-build@d844b81

@cpatulea our rpath issues are discussed in great detail by a meson developer here: #4085 (comment)

When I started with a completely clean attempt at compiling, I was receiving this message:

Length: 1017732 (994K) [application/x-xz]
Saving to: ‘/Users/cardamone/gtk/source/pkgs/nasm-2.16.01.tar.xz’

/Users/cardamone/gtk/source/pkgs/nasm-2.16.01 100%[================================================================================================>] 993.88K  4.73MB/s    in 0.2s

2024-01-05 11:48:43 (4.73 MB/s) - ‘/Users/cardamone/gtk/source/pkgs/nasm-2.16.01.tar.xz’ saved [1017732/1017732]

xzcat -d "/Users/cardamone/gtk/source/pkgs/nasm-2.16.01.tar.xz" | tar xf -
*** Error during phase checkout of nasm: Failed to find patch: nasm-2.16.01-warnings.c-needs-srcdir-patch *** [6/137]

Turns out that the patch file wasn't pulled down for me. I assumed that starting from the Setup gtk-osx step here would have taken care of that. Instead I had to curl -o nasm-2.16.01-warnings.c-needs-srcdir.patch https://github.com/Xpra-org/gtk-osx-build/blob/master/patches/nasm-2.16.01-warnings.c-needs-srcdir.patch in /Users/cardamone/Source/jhbuild/patches. I don't know if I am doing something wrong, but those changes didn't get pulled in for me automatically.

gcardamone commented 9 months ago

I am still running into the Pixman compile issues. I've modified bootstrap.modules to use 0.42.2 of pixman, along with the neon disable flags:

/Users/cardamone/Source/jhbuild/modulesets/bootstrap.modules:167:  <autotools id="pixman" autogen-sh="configure" autogenargs="--disable-gtk --disable-mmx --disable-arm-a64-neon --disable-arm-neon">
/Users/cardamone/Source/jhbuild/modulesets/bootstrap.modules:168:    <branch repo="cairo" module="pixman-${version}.tar.gz" version="0.42.2"
/Users/cardamone/Source/jhbuild/modulesets/bootstrap.modules:246:        <dep package="pixman"/>

I am still seeing the same compile time error. When I drop to a shell to investigate, I see the following:

./bootstrap.modules-:167:  <autotools id="pixman" autogen-sh="configure">
./bootstrap.modules-:168:    <branch repo="cairo" module="pixman-${version}.tar.gz" version="0.38.0"
./bootstrap.modules-:246:        <dep package="pixman"/>
./gtk-osx.modules:192:  <autotools id="pixman"
./gtk-osx.modules:195:    <branch module="releases/pixman-0.42.2.tar.gz"
./gtk-osx.modules:215:      <dep package="pixman" />
./build/pkg-config-0.29.2/check/Makefile:349:   gtk/pixman-1.pc \

Why does the bootstrap.modules file still show v0.38.0 and why are my additional autogenargs missing?

totaam commented 9 months ago

@gcardamone @cpatulea has fixed the nasm issue in https://github.com/Xpra-org/gtk-osx-build/pull/33/commits/d29f18a6d63d6702f68494ef680ee371205ff7fd + https://github.com/Xpra-org/gtk-osx-build/pull/33/commits/58152c8da488dcb792925cf60efb03f90408506e

Why does the bootstrap.modules file still show v0.38.0 and why are my additional autogenargs missing?

jhbuild updates the moduleset files on every run. You want to use local modulesets: https://github.com/Xpra-org/gtk-osx-build/pull/33/commits/c781ec5b8c993315eb126ba7d25f6e3fcbf63962