Closed totaam closed 9 years ago
totaam
commented
10 years ago Started work on this here is the first start. Hoping these can be added to svn somewhere.
Still needs work for dependencies however they all seem to build nicely. Only one patch was needed for "nasm" to support make install DESTDIR= or jhbuild complains.
I'm attaching the first files to this ticket.
totaam
commented
10 years ago xpra.modules (11.4 KiB)Xpra Moduleset
totaam
commented
10 years ago nasm-makefile-destdir.patch (0.9 KiB)totaam
commented
10 years ago Added these in r5829
After doing
jhbuild bootstrap jhbuild build jhbuild buildone gstreamer jhbuild buildone gst-plugins-base jhbuild buildone gst-plugins-goodYou should be able to now cd to
trunk/osx/jhbuildRun
jhbuild -m ./xpra.modules build meta-osx-xpra-depsThis should build and install everything else necessary to build xpra..
Currently it is missing (py)gtkglext which is the next step to be worked on.
Optional if you want subversion updated and installed you can try this as well
jhbuild -m ./xpra.modules build meta-xpra-subversion
totaam
commented
10 years ago Very good stuff, what a timesaver overall! I've updated the winswitch osx build page to refer to this moduleset.
For reference here's a link to the [/browser/xpra/trunk/osx/jhbuild jhbuild directory]
[[BR]]
Only found a few minor things that we want to tweak / note:
- I've bumped the version of
Pillow(2.3.1) andlz4(0.6.1) in r5882- we should use
httpsfor the download links and/or add checksums for the downloads- GTK and pygobject need patching (probably patch the module before building):
- include this patch to fix some warnings, at least until upstream finally merges it (if ever?)
- include this patch for fullscreen support (see #496)
- gstreamer update: we should either get gtk-osx upstream to bump the gstreamer version to the latest 0.10, carry a patch against the moduleset, or duplicate the gstreamer modules - no biggie
libpng,libtiffandlibjpeg: see #544PyOpenGLshould be bumped to 3.1 beta (see #531), using the same download or build arguments assetuptoolsif needed (in thesetup.cfg?), I've just recorded it as I did the upgrade:Downloading https://pypi.python.org/packages/source/P/PyOpenGL/PyOpenGL-3.1.0b1.zip#md5=98cf868fac68e57d1712bc8b52bc8a4b Processing PyOpenGL-3.1.0b1.zip Writing /tmp/easy_install-18IUtE/PyOpenGL-3.1.0b1/setup.cfg Running PyOpenGL-3.1.0b1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-18IUtE/PyOpenGL-3.1.0b1/egg-dist-tmp-L7SbV6 (...) Downloading https://pypi.python.org/packages/source/P/PyOpenGL-accelerate/PyOpenGL-accelerate-3.1.0b1.zip#md5=8db7b69ba0553ca5b927fe2ddbf2424e Processing PyOpenGL-accelerate-3.1.0b1.zip Writing /tmp/easy_install-33_Ewg/PyOpenGL-accelerate-3.1.0b1/setup.cfg Running PyOpenGL-accelerate-3.1.0b1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-33_Ewg/PyOpenGL-accelerate-3.1.0b1/egg-dist-tmp-7MQaJK- important: I had problems resolving
pypi.douban.comfrom here (today?) and even downloading from my UK servers it was pretty slow too at times (5KB/s at some point), which made building the python bits really tedious. Not sure if there's a better mirror we could/should be using? (easy_installmust be using a different download domain, because it didn't have any problems)- for sure, we should not be using
pypi.douban.comfor things likeCython(and probably others too) which have known good upstream download locationseasy_installwill automagically grab the latest release for us, whereas the moduleset will require updating everytime one of the dependencies is updated. I'm OK with that, it will make the upgrades more obvious (we'll have a commit reference). The old macosx build page has links to most of the dependencies to make it easier to check for new releases, we should probably do the same for the python dependencies, maybe here: [old macosx build page](http: winswitch.org dev macosx-manual.html) has links to most of the dependencies to make it easier to check for new releases, we should probably do the same for the python dependencies, maybe here: [Building](old macosx build page](http:--winswitch.org-dev-macosx-manual.html) has links to most of the dependencies to make it easier to check for new releases, we should probably do the same for the python dependencies, maybe here: [../wiki/Building) or in a new wiki page- the
ffmpegbuild is the winswitch one, it should be the xpra minimal one instead, ideally we could have both in the module somehow (unfortunately, I don't think that jhbuild does "virtual packages" ala portage)- nice to have: add gdb (not sure how to handle the
chgrpandchmodat the end..)- it would be nice not to have to use the
--insecuredownload flag at all to bootstrap the process, but I can't think of an easy way..- I seem to have a broken (py)gtkglext (#226) again now... can't wait to see that streamlined!
totaam
commented
10 years ago
- added libpng, libjpeg-turbo, libtiff in r6068
- added meta module named
meta-osx-image-libsfor easier building of the 3 libsI've had mixed results with not using --insecure or -k with curl in the end I ended up having to add the flag to the tarball.py within jhbuild to get by some issues.
totaam
commented
10 years ago
- PyOpenGL and PyOpenGL version bump in r6090 built both no problem still needs testing
totaam
commented
10 years ago New updates:
For ffmpeg, I am now using a better, more limited configure line (also on win32):
./configure --prefix=${JHBUILD_PREFIX} \ --cpu=i686 --enable-runtime-cpudetect --enable-pic --enable-memalign-hack \ --enable-static --enable-shared --enable-gpl \ --disable-everything \ --enable-swscale --enable-libx264 --enable-decoder=h264 \ --enable-libvpx --enable-decoder=vp8 --enable-decoder=vp9 --enable-decoder=hevc \ --disable-protocol=tcp --disable-protocol=rtp \ --disable-filter=aformat --disable-filter=crop --disable-filter=setpts \ --disable-filter=anull --disable-filter=format --disable-filter=trim \ --disable-filter=atrim --disable-filter=null \ --disable-programs --disable-avfilterIt took a lot of fiddling and trial and error:
--disable-allsounds like what we want... but it doesn't work: nothing gets installed- we now get problems with assembler optimizations... maybe we should move to OSX Lion as a build platform and target 64-bit only in the future?
totaam
commented
10 years ago Added patch for gtkglext in r6096
Can use it by doing something like this
git clone git://git.gnome.org/gtkglext cd gtkglext curl -O http://xpra.org/svn/Xpra/trunk/osx/jhbuild/xpra-gtkglext.patch patch -p1 <xpra-gtkglext.patch autoreconf -fiv ./configure --prefix=${JHBUILD_PREFIX} make && make installThis isn't ideal but jhbuild doesn't allow me to apply a patch when pulling source from git only when using tarballs
Next is pygtkglext.
totaam
commented
10 years ago Added
python-macholib python-modulegraph python-altgraphin r6098
As I was unable to build without these perhaps easy_install pulls these in for us normally?
After compiling and installing these I was able to build and package xpra.
Installing pygtkglext is still quite a hack as the installing portion usually fails and requires you to install them manually into site-packages/
totaam
commented
10 years ago New ones:
ORC0.4.19 is out and is worth upgrading to as it fixes memory leaks. download linklibjpeg-turbo1.3.1 is out. download linknumpy1.8.1 is outThese are going to keep accumulating until this ticket is closed..
As for the question above:
macholib,modulegraphandaltgraphare normally pulled in automatically when installingpy2appviaeasy_install.
totaam
commented
10 years ago totaam
commented
10 years ago pygtkglext-git.patch (17.7 KiB)patch for building git version of pygtkglext
totaam
commented
10 years ago The patch above builds
pygtkglextfor me using:PYTHONPATH=$JHBUILD_PREFIX/share/pygobject/2.0/:$PYTHONPATH python2.7 \ ./setup.py buildWe cannot use
setup.pyfor installing (see #226 for details), so we install it withrsync(ugly):rsync -rpvlgto build/lib.macosx*/gtk/* $JHBUILD_PREFIX/lib/python2.7/site-packages/gtk-2.0/gtk/Note: be aware of #563#comment:14 when choosing which version to use (git tree vs 1.1 snapshots)
totaam
commented
10 years ago gtkglext-1.2.0.osx-hack.tar.gz (1340.3 KiB)1.2 snapshot for osx found here: http://lists.gnu.org/archive/html/bug-gnubg/2011-04/msg00006.html
totaam
commented
10 years ago Newer versions should be updated in the moduleset:
Note to smo: work is going to keep piling on until you close this ticket...
totaam
commented
10 years ago updated versions in r6479
Compiling this whole moduleset to make sure its still working. I will use the patch that I made against gtkglext this time but should I be using this new hack that you attached? Have you tried this?
totaam
commented
10 years ago The 1.2.0 patched version I posted is a newer version with some osx patches already applied, in particular the gtk>=2.20 compatibility patches. I haven't found any regressions, so I think we should use this rather than patching an older version ourselves.
totaam
commented
10 years ago totaam
commented
10 years ago And more updates will continue to accumulate until this ticket is closed..
Today:
totaam
commented
10 years ago PyOpenGL version update in r6691 Python 2.7.7 I also changed the url in .jhbuildrc-custom and recompiled everything with no issues.
I have some modifications that work well for this as well that should be updated in the winswitch osx dev page.
Can we host gtkglext and the patch for pygtkglext somewhere on xpra.org so that I can put those into the moduleset as well?
Was your pygtkglext patch for the tarball?
totaam
commented
10 years ago Can we host gtkglext and the patch for pygtkglext somewhere on xpra.org so that I can put those into the moduleset as well? [[BR]] Sure, point me to them and I'll upload them. [[BR]] Was your pygtkglext patch for the tarball? [[BR]] Which one?
totaam
commented
10 years ago They were both attachments to this ticket that you posted before however as I see now the pygtkglext patch is for git which won't work with jhbuild.
We can however put the hacked copy of gtkglext in the moduleset and that removes one more thing.
totaam
commented
10 years ago totaam
commented
10 years ago Added gtkglext-1.2.0.osx-hack.tar.gz in r6720
had to configure with
--enable-gtk-docormake installfailed with missing html doc file
totaam
commented
10 years ago I've had to pull the 0.12.4 OSX build because pygtkglext is now completely broken (not that I could easily have noticed since I don't have GL in my macosx vms...), rebuilding it against the latest gtk+ update does not seem sufficient to make it run, see #593. You can easily get a crash just by trying to import gtkgl:
python -c "from gtk.gtkgl import *"The error seems to be the one mentioned here Fatal Python error: can't initialize module gtk.gtkgl but I can't see anything wrong, this same source code used to build and run fine...
totaam
commented
10 years ago I've just reviewed the GTK+ changes since 2.24.16, and we want quite a few of those (many are also relevant to win32 rebuilds discussed in #263#comment:10):
- Quartz: Cursor doesn't reset when the mouse leaves a toplevel window (2.24.21)
- Text input is too slow and some keys are broken after changing input source (2.24.20)
- OS X: Various input handling fixes: (2.24.21)
- Avoid a crash on first key press (I have seen this one before)
- Fall back from native input method to GtkIMContextSimple
- Avoid beeping when using native input method
- Reset preedit state when needed
- null pointer dereferenced due to multiple bugs in gtkclipboard-quartz.c (2.24.23)
totaam
commented
10 years ago r6897 updates many libraries
I had issues trying to rebuild after this as gst-plugins-ugly links against x264
jhbuild -m ./xpra.modules uninstall gst-plugins-ugly-xpra jhbuild -m ./xpra.modules -f buildone gst-plugins-ugly-xpraShould be enough to get it building properly again.
totaam
commented
10 years ago totaam
commented
10 years ago It's taken a while but I think we will soon be able to close this ticket!
Please update as discussed:
- Python 2.7.8
- PyOpenGL 3.1 and PyOpenGL_accelerate 3.1: had to do the latter by hand as easy_update failed to build it for some reason (not investigated)
- Pillow 2.5.1 (and check webp is enabled)
- ffmpeg 2.3
- x264 stable latest
- py2app 0.8.1
- gdb 7.8 which as Betters Python Scripting
- gtk+ 2.24.24
- gdk-pixbuf 2.30.8
- pango 1.36.5
- glib 2.40
- gmp 6.0.0 - important: please specify a build host type to prevent building cpu optimizations which may not be available at runtime
- setuptools 5.4.1
- libpng 1.6.12
- pyobjc 3.0.1 (worth a try - first update in almost 2 years, should be a drop in replacement, more lightweight too)
Finally, you can find three patches which allow us to build (py)gtkglext from git which I have added to svn in r6940 + r6941:
gtkglext: xpra-gtkglext.patch and gtkglext-osx-quartztagfix.diffgit clone git://git.gnome.org/gtkglext cd gtkglext curl -O "http://xpra.org/trac/browser/xpra/trunk/osx/jhbuild/xpra-gtkglext.patch?rev=6940" patch -p1 < xpra-gtkglext.patch autoreconf -fiv curl -O "http://xpra.org/trac/browser/xpra/trunk/osx/jhbuild/gtkglext-osx-quartztagfix.diff?rev=6940" patch -p1 < gtkglext-osx-quartztagfix.patch ./configure --prefix=${JHBUILD_PREFIX} --with-gdktarget=quartz make && make installpygtkglext: pygtkglext-osx-v4.patchgit clone git://git.gnome.org/pygtkglext cd pygtkglext curl -O "http://xpra.org/trac/browser/xpra/trunk/osx/jhbuild/pygtkglext-osx-v4.patch?rev=6940" patch -p1 < pygtkglext-osx-v4.patch ./autogen.sh --prefix ${JHBUILD_PREFIX} make && make installIf needed for the moduleset, we can host the patches in a more canonical location, or even make a patched source archive with the patches already applied.
Note: the pygtkglext patch is not so different from the previous version (which may still work?), and I've seen weird things happen as I rebuilt the libraries. I believe jhbuild leaves some crud behind and we must forcibly remove the build directories before trying to rebuild a library. Something to keep in mind.. since I don't see a way of forcing jhbuild to completely remove a build directory.
It may be worthwhile adding a switch somewhere so we can get debug builds for all those libraries. Not sure exactly how to add
-gto theCFLAGShowever.Once that's all done and tested, please re-assign to me so I can close this.
totaam
commented
10 years ago Dammit, looks like we're not the first ones to hit this "
invalid drawable" bug (recorded in #563#no4 - which is closed, so moving here):
- Qt bug 35342: fix (avoids window geometry 0x0)
- Tons of "invalid drawable" messages on the console (chromium bug)
- firefox bug: fix (set child view on widget)
- How do I detect “invalid drawable” in Mac OS X? (generic info)
But for gtkgl on osx, the best I found is this (which has a dead link in it - it should point here instead: GtkGLExt OS X Quartz hack patch):
I think I see it just by running
gl_checkwhich should help in testing a fix.Running the examples in
gtkglextworks without major problems, but the examples inpygtkglextshow theinvalid drawableproblem. For the record, we can also buildpygtkglext-1.1.0with the patch above, ignoring the changes toautogen.shand then using:CFLAGS="-I/usr/X11/include" ./configure --prefix=${JHBUILD_PREFIX}But the bug remains..
We can also build
gtkglextwith debug enabled:--enable-debug=[no/minimum/yes]
totaam
commented
10 years ago totaam
commented
10 years ago Edited comment:25 with instructions I have tested repeatedly from scratch, this seem to fix the "
invalid drawable" bug from #563#no4, now tracked in #593#comment:4 (at least it works on my 10.5 and 10.6 test machines so far)Let's try to close this ticket whilst things work OK! (so raising to blocker)
If jhbuild doesn't allow us to apply the
quartztagfixpatch after theautoreconfstep, we can just generate our own archive from git and stick it in svn.
totaam
commented
10 years ago gtkglext-8c13cc4-20110529.tar.bz2 (4017.1 KiB)gtkglext git snapshot
totaam
commented
10 years ago pygtkglext-896582f-20100112.tar.bz2 (469.3 KiB)pygtkglext git snaptshot
totaam
commented
10 years ago I've updated most of the stuff from comment:25
I've had to make a copy of the modulesets-stable to our source control so that I can modify them to update a few modules. We can contribute this upstream when we are satisfied everything is working.
r7184 brings in all the necessary changes to our moduleset r7186 updates many libraries to current versions
I still have to build gtkglext and pygtkglext by hand but the instructions work nicely along with the patches. We may have to make source tarballs for these to apply the patches.
I've tried to find a nicer way to fix autoconf for gtkglext so we don't have to do the second patch but didn't figure it out yet.
I've tested building all of these on an mac mini running osx 10.6.8 64bit hence all the build switches I commited for 32bit builds
I am now in the process of building this all again on a mac mini running 10.5.8 32bit but I assume it will work even easier than the other machine.
So again we are very close to closing this ticket but still some work needs to be done.
This is what my .jhbuildrc-custom looks like at the moment on the 32bit machine
module_autogenargs['cairo'] = '--enable-ft=no' setup_sdk(target="10.5", sdk_version="10.5", architectures=["i386"]) os.environ["CC"] = "/usr/bin/gcc-4.2" os.environ["DYLD_LIBRARY_PATH"] = "" build_policy = "updated-deps" modules = [ "python", "meta-gtk-osx-bootstrap", "meta-gtk-osx-core", "libcroco", "librsvg", "meta-gtk-osx-python", "meta-gtk-osx-themes", "gtk-quartz-engine", "gtk-mac-integration-python" ] #skip iconv and add some switches to turn off stuff that was breaking skip.append("libiconv") append_autogenargs('gstreamer', '--enable-introspection=no') append_autogenargs('gst-plugins-base', '--enable-introspection=no') append_autogenargs('gst-plugins-good', '--enable-introspection=no --disable-deinterlace') #change moduleset moduleset="http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/gtk-osx.modules"
totaam
commented
10 years ago More problems it doesn't use bootstrap.modules from that location it only uses it from local
~/Source/jhbuild/modulesets/bootstrap.modulesWhich is troublesome as we need to update gtk-doc so that we can build new glib if memory serves right I will also need another package to build new gtk-doc
I checked
bootstrap.modulesagainst the one locally and the only change was the change I made so its safe to use this one to edit for now.Unfortunatly I need to just copy over the one that exists with ours to do this not ideal
cd ~/Source/jhbuild/modulesets && curl -O http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/bootstrap.modules
Then run jhbuild bootstrap
totaam
commented
10 years ago Ran into another problem
gtk-doc requires itstool but itstool requires python
configure: error: itstool not found *** Error during phase configure of gtk-doc: ########## Error running ./configure --prefix /Users/blank/gtk/inst --libdir '/Users/blank/gtk/inst/lib' --disable-scrollkeeper --with-xml-catalog=$JHBUILD_PREFIX/etc/xml/catalog *** [16/19]jhbuild buildone python jhbuild buildone libxml2 jhbuild buildone itstool jhbuild bootstraplets me get past this point but I guess python, libxml2 and itstool need to move to bootstrap? Or build gtk-doc without itstool?
After this I get
*** success *** [19/19]That is enough to move on to the next steps
totaam
commented
10 years ago totaam
commented
10 years ago I'll add the md5sums to the missing packages. Looks like we still need a patch for GTK for dual monitor support I found that out today.
As for (py)gtkglext I really think we should probably just make snapshots of the repos and host our own .tar.bz2 for those 2 so we can use them with jhbuild.
What would be a good place to stick these?
totaam
commented
10 years ago
- the OSX multi-monitor fullscreen patch can be found here: gnome bug 556007 (from #496)
- I've uploaded the two git snapshots attached to this ticket to the xpra source directory: [https://www.xpra.org/src/] :
totaam
commented
10 years ago I found a way to have jhbuild pull our patch and run it after autoreconf however I can't seem to build from the gtkglext git snapshot in comment:33
I always end up with a build failure like this
i686-apple-darwin9-gcc-4.2.1: .libs/gdkglversion.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglinit.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglquery.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglconfig.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglcontext.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkgldrawable.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglpixmap.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglwindow.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglglext.o: No such file or directory make[4]: *** [libgdkglext-quartz-1.0.la] Error 1 make[3]: *** [all-recursive] Error 1 make[2]: *** [all] Error 2 make[1]: *** [all-recursive] Error 1I followed the instructions from comment:25 just to be sure and it works no problem so i'm thinking maybe that snapshot is bad?
totaam
commented
10 years ago I doubt the snapshot is bad, but it's possible.
Those errors are about object files, which are meant to be generated during the build, so a bad snapshot shouldn't be causing this unless it messed with the build scripts.
totaam
commented
10 years ago totaam
commented
10 years ago [https://pypi.python.org/pypi/Pillow/2.5.3] low priority: only CVE entries for things we don't use.
totaam
commented
10 years ago gtkglext-1.2.0.tar.bz2 (460.8 KiB)gtkglext snapshot from git works for building osx with patches
totaam
commented
10 years ago r7472 adds gtkglext and pygtkglext
The source tarball for gtkglext is one I attached to this ticket so this should probably change.
I made this one by doing this
git clone git://git.gnome.org/gtkglext && cd gtkglext git archive --prefix=gtkglext-1.2.0/ -o ../gtkglext-1.2.0.tar HEAD cd .. && bzip2 gtkglext-1.2.0.tarSeems to build now with that snapshot i'm not sure what the difference is. Feel free to make a new one and just provide me with the url and I will update it.
I will now delete my
~/gtkdirectory and start this all from scratch hopefully now it will just build everything we need.
totaam
commented
10 years ago I've uploaded your snapshot in the source dir: [http://xpra.org/src/gtkglext-1.2.0.tar.bz2]
totaam
commented
10 years ago Note: Cython should be updated to 0.21.x (I have reported fairly big bug, they may well do a point release before too long).
I have created a separate ticket for OSX GTK3: #680.
totaam
commented
10 years ago Many new changes seems to work quite well now. Still needs some polish but pretty happy with it now.
You'll need the .jhbuildrc-custom from r7651 which includes our modulesets from by url.
r7642 adds gtk-doc from bootstrap.modules to our moduleset as we need itstool and libxml2 which requires python to build it now.
r7648 adds a patch for liboil to fix autreconf from breaking found it here https://github.com/GNOME/frogr/tree/master/osx/jhbuild/patches
r7652 Cython 0.21
You need to overwrite
~/Source/jhbuild/modulesets/bootstrap.moduleswith ours I haven't found any way to use ours instead.If everything is in place I'm able to do
jhbuild bootstrap jhbuild build jhbuild build meta-subversion-xprabootstrap has 17 steps and build has 88
I had to modify
~/Source/jhbuild/jhbuild/versioncontrol/tarball.pyto add -k to curl as a lot a few https tarballs were failing since the certificates on this old mac are outdated.
Issue migrated from trac ticket # 533
component: packaging | priority: blocker | resolution: fixed | keywords: osx focus
2014-03-14 13:06:01: totaam created the issue