Closed totaam closed 3 years ago
build-1.0-with-newer-system.patch
(1.2 KiB)patch to allow us to build 1.0.x versions on a newer build system (libraries renamed or just not installed)
All built OK except for ffmpeg which tried to link with "-framework CoreMedia
", removing all those from the configure script allowed it to build just fine.
beta 1.0.6 packages posted: [http://xpra.org/beta/osx].
@smo: how do we patch the configure script? (ideally using sed
rather than a patch which may not apply cleanly with updates)
Important update from upstream: Removal of support for obsolete MacOS X Versions and global module update: ''I've completed the removal of support for versions of MacOS X earlier than 10.8 from gtk-osx. If you still need to build for 10.4u or 10.5, or on 10.4-10.7, I've tagged the last commit that supports that with "Last-Lion" to make it easy to have a jhbuildrc and modulesets to support your needs.''
I had to fix the gstreamer 1.12.0 osxaudio
compilation in gst-plugins-good
to avoid this error:
$ gst-launch-1.0 audiotestsrc ! osxaudiosink
(gst-plugin-scanner:86177): GStreamer-WARNING **: Failed to load plugin \
'/Users/osx/gtk/inst/lib/gstreamer-1.0/libgstosxaudio.so': dlopen(/Users/osx/gtk/inst/lib/gstreamer-1.0/libgstosxaudio.so, 2): \
Symbol not found: _AudioComponentFindNext
Referenced from: /Users/osx/gtk/inst/lib/gstreamer-1.0/libgstosxaudio.so (which was built for Mac OS X 10.11)
Expected in: /System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox
in /Users/osx/gtk/inst/lib/gstreamer-1.0/libgstosxaudio.so
(gst-plugin-scanner:86177): GStreamer-WARNING **: Failed to load plugin \
'/Users/osx/gtk/inst/lib/gstreamer-1.0/libgstosxaudio.so': dlopen(/Users/osx/gtk/inst/lib/gstreamer-1.0/libgstosxaudio.so, 2): \
Symbol not found: _AudioComponentFindNext
Referenced from: /Users/osx/gtk/inst/lib/gstreamer-1.0/libgstosxaudio.so (which was built for Mac OS X 10.11)
Expected in: /System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox
in /Users/osx/gtk/inst/lib/gstreamer-1.0/libgstosxaudio.so
WARNING: erroneous pipeline: no element "osxaudiosink"
I fixed this by adding -framework AudioUnit -framework CoreAudio -framework AudioToolbox
to the LDFLAGS in the sys/osxaudio/Makefile
. (inspired by OSX 10.6 gnuradio: Audio Library Error)
Not sure why this has come up now. And then... things get worse: most of the muxers no longer work: ogg and mka formats all fail with the dreaded error:
sound output pipeline error: gst-stream-error-quark: Internal data stream error. (1)
sound output appsrc do-timestamp=1 name=src emit-signals=0 block=0 is-live=0 stream-type=0 format=4 ! \
matroskademux ! opusdec ! audioconvert ! audioresample ! volume name=volume volume=0 ! \
queue name=queue min-threshold-time=0 max-size-buffers=0 max-size-bytes=0 max-size-time=450000000 leaky=2 ! \
osxaudiosink async=True qos=True sync=False
sound output gstbasesrc.c(2939)
sound output void gst_base_src_loop(GstPad *) ()
sound output /GstPipeline:pipeline0/GstAppSrc:src:
sound output streaming stopped, reason not-negotiated (-4)
Which is why this was reverted.
Ended up starting from scratch again because gstreamer failed again at runtime.
- got some cert issues, note to self: install cacert.pem before re-building (cacert.pem)
- pycairo was needed for pygobject3 (and it needed python3) - still failed, workaround here: Building gtk+3 on macOS - pygobject3 build error
- we should add brotli
Panic is over, things build fine again, provided that:
- don't run jhbuild from its own shell - nesting doesn't work anymore (should file ticket upstream)
- make sure that the
.jhbuildrc-custom
customization looks more or less like this one:_gtk_osx_use_jhbuild_python = True
setup_sdk(target="10.10", sdk_version="10.10", architectures=["x86_64"]) os.environ["CC"] = "/usr/bin/gcc" os.environ["DYLD_LIBRARY_PATH"] = "" build_policy = "updated-deps"
modules = [ "openssl", "python", "yasm", "nasm", "libxml2", "itstool", "gtk-doc", "meta-gtk-osx-bootstrap", "meta-gtk-osx-core", "libpng", "python-setuptools-scm", "libcroco", "librsvg", "meta-gtk-osx-python", "meta-gtk-osx-themes", "gtk-quartz-engine", "gtk-mac-integration-python", "meta-osx-xpra-deps", "meta-subversion-xpra", "meta-osx-xpra-pkgtools"]
moduleset="http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/gtk-osx.modules"
os.environ["SSL_CERT_FILE"] = "/Users/osx/gtk/inst/etc/ssl/cacert.pem"
(some of the modules should be picked up automatically but for some unknown reason aren't: "python-setupttols-scm" should get built before "python-lz4" automatically for example) Confirmed that the gstreamer problems occur when switching to 1.12.0, though it could still be a problem with the toolchain / leftovers from the previous version. ---- [r15865](../commit/b1a675442aa4ef21761a26b8ccd085eda46b0406) adds a module for "brotli", but the Makefile doesn't have an "install" command, so this fails. I can install it by appending this to the Makefile by hand:
install : install -d $(BINDIR) $(JHBUILD_PREFIX)/bin install -m 0755 $(BINDIR)/bro $(JHBUILD_PREFIX)/bin/
But this then fails when running it from jhbuild with:
W: Ignoring uninstalled package: brotli Traceback (most recent call last): File "/Users/osx/.local/bin/jhbuild", line 32, in
jhbuild.main.main(sys.argv[1:]) File "/Users/osx/Source/jhbuild/jhbuild/main.py", line 155, in main rc = jhbuild.commands.run(command, config, args, help=lambda: print_help(parser)) File "/Users/osx/Source/jhbuild/jhbuild/commands/init.py", line 183, in run return cmd.execute(config, args, help) File "/Users/osx/Source/jhbuild/jhbuild/commands/init.py", line 52, in execute return self.run(config, options, args, help) File "/Users/osx/Source/jhbuild/jhbuild/commands/base.py", line 268, in run return build.build() File "/Users/osx/Source/jhbuild/jhbuild/frontends/buildscript.py", line 163, in build error, altphases = module.run_phase(self, phase) File "/Users/osx/Source/jhbuild/jhbuild/modtypes/init.py", line 420, in run_phase method(buildscript) File "/Users/osx/Source/jhbuild/jhbuild/modtypes/autotools.py", line 314, in do_install self.process_install(buildscript, self.get_revision()) File "/Users/osx/Source/jhbuild/jhbuild/modtypes/init.py", line 316, in process_install new_contents = fileutils.accumulate_dirtree_contents(destdir_prefix) File "/Users/osx/Source/jhbuild/jhbuild/utils/fileutils.py", line 45, in accumulate_dirtree_contents _accumulate_dirtree_contents_recurse(path, contents) File "/Users/osx/Source/jhbuild/jhbuild/utils/fileutils.py", line 27, in _accumulate_dirtree_contents_recurse names = os.listdir(path) OSError: [Errno 2] No such file or directory: '/Users/osx/gtk/inst/_jhbuild/root-brotli/Users/osx/gtk/inst' @smo: how do I make it play nice?
*** Installing brotli *** [1/1] make DESTDIR=/Users/spikesinc/gtk/inst/_jhbuild/root-brotli install
Have to install to DESTDIR instead and that should work
Easier way is to just use cmake it would seem
I had a few issues myself when building all of this from the start again. First off I change my .jhbuildrc-custom to be like yours and downloaded the cacert.pem. This worked nicely.
*** Checking out glib *** [20/134] curl --continue-at - -L http://ftp.gnome.org/pub/GNOME/sources/glib/2.48/glib-2.48.0.tar.xz -o /Users/spikesinc/gtk/source/pkgs/glib-2.48.0.tar.xz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 377 100 377 0 0 577 0 --:--:-- --:--:-- --:--:-- 576 100 7312k 100 7312k 0 0 1727k 0 0:00:04 0:00:04 --:--:-- 2685k xzcat -d "/Users/spikesinc/gtk/source/pkgs/glib-2.48.0.tar.xz" | tar xf - *** Error during phase checkout of glib: could not download patch (error: 404) *** [20/134]
Ran into one of the patches missing so I will have to look into what that was. I was able to just start the build again and continue.
I also had issues where some of the compiling processes would get killed? Restarting the build usually got me by these issues.
python-setuptools-scm tried to install before setuptools was installed which caused it to produce this stacktrace
Traceback (most recent call last): File "setup.py", line 15, in <module> import setuptools ImportError: No module named setuptools
Also came across this error trying to install python-zeroconf not sure what to do about this one.
*** Building python-zeroconf *** [87/134] python setup.py build Traceback (most recent call last): File "setup.py", line 15, in <module> [l for l in open(join(PROJECT_ROOT, 'zeroconf.py')) if '__version__' in l][0] LookupError: unknown encoding: *** Error during phase build of python-zeroconf: ########## Error running python setup.py build *** [87/134]
I guess that was too easy. Still have this issue with brotli
[JH][~/xpra-trunk/osx/jhbuild]$ bro dyld: Library not loaded: libbrotlienc.0.6.0.dylib Referenced from: /Users/build/gtk/inst/bin/bro Reason: image not found Trace/BPT trap: 5
I don't have my macos VM with me so I cannot test today, but maybe the cmake build is different (I don't think the pure "make" build uses shared libraries), and maybe the shared library isn't installed, or not properly installed?
I also had issues where some of the compiling processes would get killed? Restarting the build usually got me by these issues. OOM killer maybe? I'm giving this build VM lots of cpus and tons of ram..
python-setuptools-scm tried to install before setuptools was installed which caused it to produce this stacktrace Adding "python-setuptools" before "python-setuptools-scm" in jhbuildrc-custom should fix that.
Also came across this error trying to install python-zeroconf not sure what to do about this one. Weird one. I can't reproduce. Could be something in your environment which is different.
I guess that was too easy. Still have this issue with brotli Are you trying to run it from within a jhbuild shell? (I got the same error when I tried from outside the jhbuild shell env)
Updates (all safe to backport and recommended):
- r16002: libtiff 4.0.8
- r15998: python-lz4 0.9.4, netifaces 0.10.6
- r16003 + r16004: expat-2.2.0 + lame patch
- r16027: opus
- r16029: openssl 1.1.0f
TODO:
- lz4 is broken:
*** Building python-lz4 *** [99/122] python setup.py build Traceback (most recent call last): File "setup.py", line 161, in <module> 'Programming Language :: Python :: 3.6', File "/Users/osx/gtk/inst/lib/python2.7/distutils/core.py", line 111, in setup _setup_distribution # distklass(attrs) File "/Users/osx/gtk/inst/lib/python2.7/site-packages/setuptools/dist.py", line 271, in __init__ _Distribution.__init__(self,attrs) File "/Users/osx/gtk/inst/lib/python2.7/distutils/dist.py", line 287, in __init__ self.finalize_options() File "/Users/osx/gtk/inst/lib/python2.7/site-packages/setuptools/dist.py", line 326, in finalize_options ep.load()(self, ep.name, value) File "/Users/osx/gtk/inst/lib/python2.7/site-packages/setuptools_scm/integration.py", line 19, in version_keyword dist.metadata.version = get_version(**value) File "/Users/osx/gtk/inst/lib/python2.7/site-packages/setuptools_scm/__init__.py", line 117, in get_version parsed_version = _do_parse(root, parse) File "/Users/osx/gtk/inst/lib/python2.7/site-packages/setuptools_scm/__init__.py", line 95, in _do_parse "use git+https://github.com/user/proj.git#egg=proj" % root) LookupError: setuptools-scm was unable to detect version for '/Users/osx/gtk/source/python-lz4-0.9.4'.
- tiff is broken:
*** Configuring libtiff *** [16/122] ./configure --prefix /Users/osx/gtk/inst --without-x (...) config.status: error: cannot find input file: `html/Makefile.in' *** Error during phase configure of libtiff: ########## Error running ./configure --prefix /Users/osx/gtk/inst --without-x *** [16/122]
FFS: I fixed most of the modules, see comment:15 but now we have a new packaging problem with python-cryptography (detected by the unit tests):
ERROR: unit tests failed, see /Users/osx/Xpra-svn/trunk/src/unittests/unittest.log: Traceback (most recent call last): File "/Users/osx/Xpra-svn/trunk/src/unittests/unit/net/crypto_test.py", line 30, in setUp pycryptography_backend.init() File "/Users/osx/gtk/inst/lib/python2.7/site-packages/xpra/net/pycryptography_backend.py", line 52, in init backend = default_backend() File "/Users/osx/gtk/inst/lib/python2.7/site-packages/cryptography/hazmat/backends/__init__.py", line 15, in default_backend from cryptography.hazmat.backends.openssl.backend import backend File "/Users/osx/gtk/inst/lib/python2.7/site-packages/cryptography/hazmat/backends/openssl/__init__.py", line 7, in <module> from cryptography.hazmat.backends.openssl.backend import backend File "/Users/osx/gtk/inst/lib/python2.7/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 16, in <module> from cryptography import utils, x509 File "/Users/osx/gtk/inst/lib/python2.7/site-packages/cryptography/x509/__init__.py", line 8, in <module> from cryptography.x509 import certificate_transparency ImportError: cannot import name certificate_transparency
Figuring out the new dependency:
$ python -c "from cryptography.hazmat.backends.openssl.backend import backend" Traceback (most recent call last): File "<string>", line 1, in <module> File "/Users/osx/gtk/inst/lib/python2.7/site-packages/cryptography/hazmat/backends/openssl/__init__.py", line 7, in <module> from cryptography.hazmat.backends.openssl.backend import backend File "/Users/osx/gtk/inst/lib/python2.7/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 16, in <module> from cryptography import utils, x509 File "/Users/osx/gtk/inst/lib/python2.7/site-packages/cryptography/x509/__init__.py", line 9, in <module> from cryptography.x509.base import ( File "/Users/osx/gtk/inst/lib/python2.7/site-packages/cryptography/x509/base.py", line 16, in <module> from cryptography.x509.extensions import Extension, ExtensionType File "/Users/osx/gtk/inst/lib/python2.7/site-packages/cryptography/x509/extensions.py", line 13, in <module> from asn1crypto.keys import PublicKeyInfo ImportError: No module named asn1crypto.keys
So r16045 adds
asn1crypto
to the moduleset.But then we hit a new problem:
ERROR: test_backends (__main__.TestCrypto) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/osx/Xpra-svn/trunk/src/unittests/unit/net/crypto_test.py", line 83, in test_backends self.do_test_backend() File "/Users/osx/Xpra-svn/trunk/src/unittests/unit/net/crypto_test.py", line 63, in do_test_backend v = enc.encrypt(message) File "/Users/osx/gtk/inst/lib/python2.7/site-packages/cryptography/hazmat/primitives/ciphers/base.py", line 149, in update return self._ctx.update(data) File "/Users/osx/gtk/inst/lib/python2.7/site-packages/cryptography/hazmat/backends/openssl/ciphers.py", line 116, in update n = self.update_into(data, buf) File "/Users/osx/gtk/inst/lib/python2.7/site-packages/cryptography/hazmat/backends/openssl/ciphers.py", line 127, in update_into "unsigned char *", self._backend._ffi.from_buffer(buf) TypeError: from_buffer() cannot return the address of the raw string within a str or unicode or bytearray object
Looks like we can't feed it a string or bytearray now. Discussion here: Creating a cdata from a buffer This is a CFFI issue... and we're stuck on version 1.5.2 (see Xpra-org/xpra#1404#comment:18 for details). Downgrade python-cryptography? (we can't do that forever..) Major PITA.
Trying to use ctypes to wrap the string buffer does not help:
from ctypes import bmsg = create_string_buffer( v = enc.encrypt(bmsg)
Fails with the same error message, also tried feeding it a numpy array, memoryview. Same error. WTF? Also tried upgrading CFFI to the latest version.
16044 contains many backports to the v1.0.x branch.
Reverted back to python-cryptography 1.8.1 in r16047. We're stuck with this version for now, moving this to a new ticket: Xpra-org/xpra#1544
Updates:
- r16139: opus 1.2.1
Backports: 16148 (+16154 + 16155), 16149, 16150 (+16151 + 16152)
AFAICT, all looks good - except for the
CoreMedia
linkage failure from comment:1 when building the 1.0.x branch on 10.5.x We just need a trivialsed -i -e s/-framework CoreMedia//g configure
somewhere before running configure.(and ignoring the lz4 build issue which seems to be this bug: setuptools_scm issue 164)
@smo: any ideas?
- r16187 removes python-pam (follow up from r16184) - not to be backported
- r16189 Pillow 4.2.0
- r16188 new x264 snapshot, I thought this rebuild would be enough to fix some bundler errors on libx264:
jhbuild build -f ffmpeg jhbuild build -f gst-plugins-ugly-1.0
Found those two packages by looking at all the .dylib and .so with otool, identifying those that link to libx264 and then finding the packages that contain those files.
But I had to do a full rebuild instead:
jhbuild build -f
Got a build error on cryptography:
building '_openssl' extension /usr/bin/gcc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch x86_64 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -mmacosx-version-min=10.10 -I/Users/osx/gtk/inst/include -arch x86_64 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -I/Users/osx/gtk/inst/include/python2.7 -c build/temp.macosx-10.10-x86_64-2.7/_openssl.c -o build/temp.macosx-10.10-x86_64-2.7/build/temp.macosx-10.10-x86_64-2.7/_openssl.o build/temp.macosx-10.10-x86_64-2.7/_openssl.c:745:23: error: conflicting types for 'ASN1_TIME_to_generalizedtime' ASN1_GENERALIZEDTIME *ASN1_TIME_to_generalizedtime(ASN1_TIME *, ^ /Users/osx/gtk/inst/include/openssl/asn1.h:628:23: note: previous declaration is here ASN1_GENERALIZEDTIME *ASN1_TIME_to_generalizedtime(const ASN1_TIME *t, (..) build/temp.macosx-10.10-x86_64-2.7/_openssl.c:64175:29: warning: passing 'X509_ALGOR **' (aka 'struct X509_algor_st **') to parameter of type 'const X509_ALGOR **' (aka 'const struct X509_algor_st **') discards qualifiers in nested pointer types [-Wincompatible-pointer-types-discards-qualifiers] { X509_get0_signature(x0, x1, x2); } ^~ /Users/osx/gtk/inst/include/openssl/x509.h:553:45: note: passing argument to parameter 'palg' here const X509_ALGOR **palg, const X509 *x); ^ 50 warnings and 1 error generated.
Which I will ignore for now...
The openssl issue is tracked in this ticket: fix compilation on 1.1.0f and the fix is included in 1.8.2: various backports for 1.8.2.
Built fine now, though I had to download the updated
xpra.modules
file by hand as jhbuild insisted on using the old one.. no idea why.
crickets - moving to 2.2: Xpra-org/gtk-osx-build#4
Tracker ticket for the moduleset changes and those that may need to be backported. (see Xpra-org/xpra#1404 for 2.0)
Probably safe to backport to 1.x and 2.0.x:
get_depth
in gtkglNot needed: