search

JuliaPackaging / BinaryBuilder.jl

Binary Dependency Builder for Julia
https://binarybuilder.org
Other
391 stars 100 forks source link

Qt build issues #393

Closed barche closed 4 years ago

barche commented 5 years ago

When building Qt using the following configure command from inside a BinaryBuilder.runshell(Linux(:x86_64, compiler_abi=CompilerABI(:gcc7,:cxx11)))

./configure -v -prefix /workspace/install -opensource -confirm-license -skip qtactiveqt -skip qtandroidextras -skip qtcanvas3d -skip qtconnectivity -skip qtdatavis3d -skip qtdoc -skip qtgamepad -skip qtnetworkauth -skip qtpurchasing -skip qtremoteobjects -skip qtscript -skip qtscxml -skip qtsensors -skip qtserialbus -skip qtserialport -skip qtspeech -skip qtvirtualkeyboard -skip qtwayland -skip qtwebchannel -skip qtwebengine -skip qtwebglplugin -skip qtwebsockets -skip qtwebview -release

This results in the compile command and error:

g++ -c -o qfilesystemengine_unix.o   -std=c++11 -ffunction-sections -g -g  -I/workspace/qt-everywhere-src-5.11.2/qtbase/qmake -I/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/library -I/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators -I/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/unix -I/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/win32 -I/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/mac -I/workspace/qt-everywhere-src-5.11.2/qtbase/include -I/workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore -I/workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore/5.11.2 -I/workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore/5.11.2/QtCore -I../src/corelib/global -I/workspace/qt-everywhere-src-5.11.2/qtbase/mkspecs/linux-g++ -DQT_VERSION_STR=\"5.11.2\" -DQT_VERSION_MAJOR=5 -DQT_VERSION_MINOR=11 -DQT_VERSION_PATCH=2 -DQT_BUILD_QMAKE -DQT_BOOTSTRAPPED -DPROEVALUATOR_FULL -DQT_NO_FOREACH /workspace/qt-everywhere-src-5.11.2/qtbase/src/corelib/io/qfilesystemengine_unix.cpp
/workspace/qt-everywhere-src-5.11.2/qtbase/src/corelib/io/qfilesystemengine_unix.cpp: In function 'int qt_fstatx(int, statx*)':
/workspace/qt-everywhere-src-5.11.2/qtbase/src/corelib/io/qfilesystemengine_unix.cpp:365:34: error: 'AT_EMPTY_PATH' was not declared in this scope
     return qt_real_statx(fd, "", AT_EMPTY_PATH, statxBuffer);
                                  ^~~~~~~~~~~~~
make: *** [Makefile:423: qfilesystemengine_unix.o] Error 1

On my Arch Linux machine the same command works and AT_EMPTY_PATH is defined in /usr/include/fcntl.h. In the BinaryBuilder, it is defined in /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/include/linux/fcntl.h, but it seems that header is not found. Any ideas what could cause this?

staticfloat commented 5 years ago

Hmmm, not sure why this would be happening. I think it might be helpful for you to run with --debug, then re-run that last line with -v:

$ g++ -c -o qfilesystemengine_unix.o   -std=c++11 -ffunction-sections -g -g  -I/workspace/qt-everywhere-src-5.11.2/qtbase/qmake -I/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/library -I/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators -I/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/unix -I/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/win32 -I/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/mac -I/workspace/qt-everywhere-src-5.11.2/qtbase/include -I/workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore -I/workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore/5.11.2 -I/workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore/5.11.2/QtCore -I../src/corelib/global -I/workspace/qt-everywhere-src-5.11.2/qtbase/mkspecs/linux-g++ -DQT_VERSION_STR=\"5.11.2\" -DQT_VERSION_MAJOR=5 -DQT_VERSION_MINOR=11 -DQT_VERSION_PATCH=2 -DQT_BUILD_QMAKE -DQT_BOOTSTRAPPED -DPROEVALUATOR_FULL -DQT_NO_FOREACH /workspace/qt-everywhere-src-5.11.2/qtbase/src/corelib/io/qfilesystemengine_unix.cpp -v

And any other flags that might be necessary to see which header files it's picking up.

barche commented 5 years ago

OK, the full output with -v is:

Using built-in specs.
COLLECT_GCC=g++
Target: x86_64-linux-gnu
Configured with: /workspace/srcdir/gcc-7.1.0/configure --prefix=/workspace/destdir --target=x86_64-linux-gnu --host=x86_64-alpine-linux-musl --build=x86_64-alpine-linux-musl --disable-multilib --disable-werror --disable-bootstrap --enable-host-shared --enable-threads=posix --with-sysroot=/workspace/destdir/x86_64-linux-gnu/sys-root --enable-languages=c,c++,fortran,objc,obj-c++
Thread model: posix
gcc version 7.1.0 (GCC) 
COLLECT_GCC_OPTIONS='-c' '-o' 'qfilesystemengine_unix.o' '-std=c++11' '-ffunction-sections' '-g' '-g' '-I' '/workspace/qt-everywhere-src-5.11.2/qtbase/qmake' '-I' '/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/library' '-I' '/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators' '-I' '/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/unix' '-I' '/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/win32' '-I' '/workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/mac' '-I' '/workspace/qt-everywhere-src-5.11.2/qtbase/include' '-I' '/workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore' '-I' '/workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore/5.11.2' '-I' '/workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore/5.11.2/QtCore' '-I' '../src/corelib/global' '-I' '/workspace/qt-everywhere-src-5.11.2/qtbase/mkspecs/linux-g++' '-D' 'QT_VERSION_STR="5.11.2"' '-D' 'QT_VERSION_MAJOR=5' '-D' 'QT_VERSION_MINOR=11' '-D' 'QT_VERSION_PATCH=2' '-D' 'QT_BUILD_QMAKE' '-D' 'QT_BOOTSTRAPPED' '-D' 'PROEVALUATOR_FULL' '-D' 'QT_NO_FOREACH' '-v' '-shared-libgcc' '-mtune=generic' '-march=x86-64'
 /opt/x86_64-linux-gnu/bin/../libexec/gcc/x86_64-linux-gnu/7.1.0/cc1plus -quiet -v -I /workspace/qt-everywhere-src-5.11.2/qtbase/qmake -I /workspace/qt-everywhere-src-5.11.2/qtbase/qmake/library -I /workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators -I /workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/unix -I /workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/win32 -I /workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/mac -I /workspace/qt-everywhere-src-5.11.2/qtbase/include -I /workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore -I /workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore/5.11.2 -I /workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore/5.11.2/QtCore -I ../src/corelib/global -I /workspace/qt-everywhere-src-5.11.2/qtbase/mkspecs/linux-g++ -iprefix /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/7.1.0/ -isysroot /opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root -D_GNU_SOURCE -D QT_VERSION_STR="5.11.2" -D QT_VERSION_MAJOR=5 -D QT_VERSION_MINOR=11 -D QT_VERSION_PATCH=2 -D QT_BUILD_QMAKE -D QT_BOOTSTRAPPED -D PROEVALUATOR_FULL -D QT_NO_FOREACH /workspace/qt-everywhere-src-5.11.2/qtbase/src/corelib/io/qfilesystemengine_unix.cpp -quiet -dumpbase qfilesystemengine_unix.cpp -mtune=generic -march=x86-64 -auxbase-strip qfilesystemengine_unix.o -g -g -std=c++11 -version -ffunction-sections -o /tmp/cc5RyNOZ.s
GNU C++11 (GCC) version 7.1.0 (x86_64-linux-gnu)
        compiled by GNU C version 4.8.5, GMP version 6.1.0, MPFR version 3.1.4, MPC version 1.0.3, isl version isl-0.16.1-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring duplicate directory "/opt/x86_64-linux-gnu/bin/../lib/gcc/../../lib/gcc/x86_64-linux-gnu/7.1.0/../../../../x86_64-linux-gnu/include/c++/7.1.0"
ignoring duplicate directory "/opt/x86_64-linux-gnu/bin/../lib/gcc/../../lib/gcc/x86_64-linux-gnu/7.1.0/../../../../x86_64-linux-gnu/include/c++/7.1.0/x86_64-linux-gnu"
ignoring duplicate directory "/opt/x86_64-linux-gnu/bin/../lib/gcc/../../lib/gcc/x86_64-linux-gnu/7.1.0/../../../../x86_64-linux-gnu/include/c++/7.1.0/backward"
ignoring duplicate directory "/opt/x86_64-linux-gnu/bin/../lib/gcc/../../lib/gcc/x86_64-linux-gnu/7.1.0/include"
ignoring nonexistent directory "/opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/local/include"
ignoring duplicate directory "/opt/x86_64-linux-gnu/bin/../lib/gcc/../../lib/gcc/x86_64-linux-gnu/7.1.0/include-fixed"
ignoring duplicate directory "/opt/x86_64-linux-gnu/bin/../lib/gcc/../../lib/gcc/x86_64-linux-gnu/7.1.0/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "../src/corelib/global"
#include "..." search starts here:
#include <...> search starts here:
 /workspace/qt-everywhere-src-5.11.2/qtbase/qmake
 /workspace/qt-everywhere-src-5.11.2/qtbase/qmake/library
 /workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators
 /workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/unix
 /workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/win32
 /workspace/qt-everywhere-src-5.11.2/qtbase/qmake/generators/mac
 /workspace/qt-everywhere-src-5.11.2/qtbase/include
 /workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore
 /workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore/5.11.2
 /workspace/qt-everywhere-src-5.11.2/qtbase/include/QtCore/5.11.2/QtCore
 /workspace/qt-everywhere-src-5.11.2/qtbase/mkspecs/linux-g++
 /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/7.1.0/../../../../x86_64-linux-gnu/include/c++/7.1.0
 /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/7.1.0/../../../../x86_64-linux-gnu/include/c++/7.1.0/x86_64-linux-gnu
 /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/7.1.0/../../../../x86_64-linux-gnu/include/c++/7.1.0/backward
 /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/7.1.0/include
 /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/7.1.0/include-fixed
 /opt/x86_64-linux-gnu/bin/../lib/gcc/x86_64-linux-gnu/7.1.0/../../../../x86_64-linux-gnu/include
 /opt/x86_64-linux-gnu/bin/../x86_64-linux-gnu/sys-root/usr/include
End of search list.
GNU C++11 (GCC) version 7.1.0 (x86_64-linux-gnu)
        compiled by GNU C version 4.8.5, GMP version 6.1.0, MPFR version 3.1.4, MPC version 1.0.3, isl version isl-0.16.1-GMP

GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 8bc170bbed849ac5a1b988e74043ef6f
/workspace/qt-everywhere-src-5.11.2/qtbase/src/corelib/io/qfilesystemengine_unix.cpp: In function 'int qt_fstatx(int, statx*)':
/workspace/qt-everywhere-src-5.11.2/qtbase/src/corelib/io/qfilesystemengine_unix.cpp:365:34: error: 'AT_EMPTY_PATH' was not declared in this scope
     return qt_real_statx(fd, "", AT_EMPTY_PATH, statxBuffer);

It seems to be missing /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/include/linux.

barche commented 5 years ago

It seems the AT-EMPTY_PATH was an issue with Qt, it went away when trying again with Qt 5.12. I now hit the next hurdle, which is OpenGL:

GL/gl.h: No such file or directory

Did anyone do OpenGL related stuff in BinaryBuilder yet?

barche commented 5 years ago

Update: I think I need this: https://github.com/NVIDIA/libglvnd

Unfortunately, it has libX11 as a build dependency. Is there any way to add this to the basic images, or should it also be compiled from source?

staticfloat commented 5 years ago

We will not be shipping X11 to people's computers, I don't think. We're going to need to rely on them to already have that installed. So for that, you can just apk add libx11 as a step in your build script. This will only work for Linux x86_64 (and maybe i686) of course, it will not work for any other architecture or operating system. I assume you are starting with these easier targets for now?

barche commented 5 years ago

Yes, as far as I understand the mechanisms for accessing OpenGL are different on macOS and Windows, so this is only for linux. The apk command is probably exactly what I need, completely missed that somehow, thanks :)

barche commented 5 years ago

The apk command worked to install the dependencies, but the configuration of libglvnd goes wrong, it seems to add /workspace/destdir to the prefix, e.g. in the config.log:

X11_CFLAGS='-I/workspace/destdir/usr/include '

I can fix this by running the configure script like this:

CFLAGS=-I/usr/include LDFLAGS=-L/usr/lib ./configure --prefix=$prefix --host=$target

Is that the correct workaround?

staticfloat commented 5 years ago

You want to override only X11_CFLAGS I think, since you want other things to search /workspace/destdir; try just passing the X11-flags to ./configure, e.g.:

./configure --prefix=${prefix} --host=${target} X11_CFLAGS=-I/usr/include
barche commented 5 years ago

I think adapting the flags may not be the solution after all: when testing on ARM, it fails. Is there an apk command or something similar to get the X11 libs for the target arch?

staticfloat commented 5 years ago

No; this is why I said "this will only work for Linux x86_64"; because apk add can only add for the platform it's running on. To get this to truly work for all platforms, we will need a way to supply libx11 for each platform. While it's not crazy to do this, I think it's reasonable to restrict ourselves to Linux x86_64 for now, since I don't think many people are using GUIs on ARM machines right now.

barche commented 5 years ago

Probably not many people would be inclined to try it, but I did get QML.jl to work on a Raspberry Pi, and it was surprisingly fast, so it would still be nice to have, though it is of course not a priority.

How are the arch-specific images generated? It seems apk has options to set a root and an arch, would that be a possibility?

staticfloat commented 5 years ago

They're not really arch-specific images; we build a suite of cross-compilers but those cross-compilers are themselves x86_64 executables. It's true that some of the compiler support libraries are actual ARM binaries, but there's actually very few pieces of ARM code within the BB images. We have so far been pretty much relying on ourselves building everything from scratch.

It's possible that building X11 from scratch isn't that bad. We probably won't ship it, but having it to link against at build time shouldn't be too bad...

aminya commented 5 years ago

No; this is why I said "this will only work for Linux x86_64"; because apk add can only add for the platform it's running on. To get this to truly work for all platforms, we will need a way to supply libx11 for each platform. While it's not crazy to do this, I think it's reasonable to restrict ourselves to Linux x86_64 for now, since I don't think many people are using GUIs on ARM machines right now.

What's the state of this? What about Windows?

giordano commented 5 years ago

What's the state of this?

We now have a builder for libX11 (and a few other related X libraries): https://github.com/JuliaPackaging/Yggdrasil/pull/42. Binary artifacts are in the releases of https://github.com/JuliaBinaryWrappers/X11_jll.jl

What about Windows?

Unfortunately libX11 doesn't build easily for Windows, but maybe Qt doesn't require libX11 on that platform?

barche commented 5 years ago

Great news, and indeed, Qt doesn't need X11 on Windows.

barche commented 4 years ago

Things are looking good, on my local machine cross-compile to Windows works, and for Linux the final dependencies should be in Yggdrasil soon ( see https://github.com/JuliaPackaging/Yggdrasil/pull/242 )

Next step: check with QML.jl that the resulting binaries actually work and produce pretty-looking GUIs ;)

giordano commented 4 years ago

Wow, this is exciting! I personally like the look and feel of Qt applications better than GTK ones :smiley:

aminya commented 4 years ago

Yay! This is very exciting. 😃

giordano commented 4 years ago

Out of curiosity, is there any update about building Qt with BinaryBuilder? I think that having Qt in Yggdrasil might help with building GR from source: https://github.com/JuliaPackaging/Yggdrasil/issues/11

barche commented 4 years ago

It's mostly working locally for me, I'm hesitant to upload it because I still need to test the binaries with QML.jl, which is on my holiday todo-list ;)

barche commented 4 years ago

I mistakenly thought the macOS build would be easy. What I am seeing now is very strange: the exact same command works in the debug shell, but fails in the script. Here is what I have in the shell script:

cd $WORKSPACE/srcdir
cd MacOSX*.sdk
rm -rf /opt/$target/$target/sys-root/usr/include
rm -rf /opt/$target/$target/sys-root/System
mv usr/include /opt/$target/$target/sys-root/usr
mv System /opt/$target/$target/sys-root/

cd $WORKSPACE/srcdir/qt-everywhere-src-5.13.2/qtbase

cat <<EOT > mkspecs/features/mac/default_pre.prf
CONFIG = asset_catalogs rez \$\$CONFIG
load(default_pre)

QMAKE_ASSET_CATALOGS_APP_ICON = AppIcon

macx-xcode:qtConfig(static): \\
    QMAKE_XCODE_DEBUG_INFORMATION_FORMAT = dwarf

QMAKE_XCODE_LIBRARY_SUFFIX_SETTING = QT_LIBRARY_SUFFIX

xcode_copy_phase_strip_setting.name = COPY_PHASE_STRIP
xcode_copy_phase_strip_setting.value = NO
QMAKE_MAC_XCODE_SETTINGS += xcode_copy_phase_strip_setting
EOT

sed -i '1s;^;QMAKE_MAC_SDK.macosx.Path = '"/opt/$target/$target/sys-root"'\
QMAKE_MAC_SDK.macosx.SDKVersion = '"10.13"'\
QMAKE_MAC_SDK.macosx.PlatformPath = '"/opt/$target"'\n;' 'mkspecs/features/mac/sdk.prf'

sed -i 's/-fuse-ld=x86_64-apple-darwin14/-fuse-ld=\/opt\/bin\/x86_64-apple-darwin14-ld/g' /opt/bin/x86_64-apple-darwin14-clang++
sed -i 's/-fuse-ld=x86_64-apple-darwin14/-fuse-ld=\/opt\/bin\/x86_64-apple-darwin14-ld/g' /opt/bin/x86_64-apple-darwin14-clang

cd $WORKSPACE/srcdir/build
apk add g++ linux-headers
export PATH=$(echo "$PATH" | sed -e 's/\/opt\/bin://')
QT_MAC_SDK_NO_VERSION_CHECK=1 ../qt-everywhere-src-5.13.2/configure QMAKE_CXXFLAGS+=-std=c++11 -platform linux-g++ -xplatform macx-clang -device-option CROSS_COMPILE=/opt/bin/$target- \
    __COMMON_OPTIONS__ \
    -skip qtwinextras

When running using julia --color=yes build_tarballs.jl --verbose --debug I get the following error when it checks compiler C++11 compatibility:

> /opt/bin/x86_64-apple-darwin14-clang++ -c -pipe -stdlib=libc++ -std=c++11 -O2  -arch x86_64 -isysroot /opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root -mmacosx-version-min=10.12 -w -fPIC  
-I. -I/workspace/srcdir/qt-everywhere-src-5.13.2/qtbase/mkspecs/macx-clang -o main.o main.cpp
> main.cpp:2:10: fatal error: 'random' file not found
> #include <random>
>          ^~~~~~~~
> 1 error generated.

Running the apk command and the configure command again in the debug shell allows the configure to complete, although the actual build later fails with:

/opt/bin/x86_64-apple-darwin14-ranlib -s ../../../lib/libqtpcre2.a
/opt/x86_64-apple-darwin14/bin/llvm-ranlib: /opt/x86_64-apple-darwin14/bin/llvm-ranlib takes just one archive as an argument.

So my concrete questions:

giordano commented 4 years ago

Any ideas what could be different between the debug shell and the scripted run to cause not to be found?

When you enter the debug shell, the Root file system is replaced by a fresh one. All changes made before the failure have to be repeated, like installing packages with apk, and maybe replacing the macOS SDK

barche commented 4 years ago

Ah, thanks, that explains it, I think I 'm not installing the SDK properly, removing that include dir probably destroys the STL. Time for bed now, unfortunately, will try this tomorrow.

barche commented 4 years ago

The build now starts, but now I hit "fatal error: 'IOKit/IOReturn.h' file not found".

The solution seems simple: https://bugzilla.mozilla.org/show_bug.cgi?id=1544763#c6

But I'm posting it here anyway, since I believe in sharing the pain and also it might come up in future BinaryBuilder builds.

barche commented 4 years ago

Very close now, but I need help, since this is BinaryBuilder-specific. I put libQt5Gui as a library product, but Qt builds as a Framework for macOS, so destdir/lib actually contains QtGui.framework. Is there a way to tell BinaryBuilder to detect these as a LibraryProduct?

I haven't tested yet if the missing "5" is the problem, getting to his point takes almost an hour of compiling.

For reference, the complete error:

[ Info: /home/user/src/julia/builders/qt5builder/build/x86_64-apple-darwin14/n6NNLZNx/destdir/bin/qmlscene matches our search criteria of ["qmlscene"]
[ Info: Could not locate libQt5Gui inside ["/home/user/src/julia/builders/qt5builder/build/x86_64-apple-darwin14/n6NNLZNx/destdir/lib64", "/home/user/src/julia/builders/qt5builder/build/x86_64-apple-darwin14/n6NNLZNx/destdir/lib"]
┌ Error: Built qt5 but libqt5gui still unsatisfied:
â”” @ BinaryBuilder ~/src/julia/pkgdev/BinaryBuilder/src/AutoBuild.jl:731
ERROR: LoadError: Cannot continue with unsatisfied build products!
Stacktrace:
 [1] #autobuild#314(::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Nothing, ::Bool, ::Nothing, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(autobuild), ::String, ::String, ::VersionNumber, ::Array{Pair{String,String},1}, ::String, ::Array{MacOS,1}, ::Array{Product,1}, ::Array{Any,1}) at /home/user/src/julia/pkgdev/BinaryBuilder/src/AutoBuild.jl:736
 [2] (::BinaryBuilder.var"#kw##autobuild")(::NamedTuple{(:verbose, :debug, :meta_json_stream),Tuple{Bool,Bool,Nothing}}, ::typeof(autobuild), ::String, ::String, ::VersionNumber, ::Array{Pair{String,String},1}, ::String, ::Array{MacOS,1}, ::Array{Product,1}, ::Array{Any,1}) at ./none:0
 [3] #build_tarballs#287(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(build_tarballs), ::Array{String,1}, ::String, ::VersionNumber, ::Array{Pair{String,String},1}, ::String, ::Array{MacOS,1}, ::Array{Product,1}, ::Array{Any,1}) at /home/user/src/julia/pkgdev/BinaryBuilder/src/AutoBuild.jl:187
 [4] build_tarballs(::Array{String,1}, ::String, ::VersionNumber, ::Array{Pair{String,String},1}, ::String, ::Array{MacOS,1}, ::Array{Product,1}, ::Array{Any,1}) at /home/user/src/julia/pkgdev/BinaryBuilder/src/AutoBuild.jl:24
 [5] top-level scope at /home/user/src/julia/builders/qt5builder/build_tarballs.jl:147
in expression starting at /home/user/src/julia/builders/qt5builder/build_tarballs.jl:147
giordano commented 4 years ago

You should have got a message above telling why libQt5Gui can't be dlopen'ed, probably saying that there is a missing library. Could you please post that error, too?

barche commented 4 years ago

You should have got a message above telling why libQt5Gui can't be dlopen'ed

No, it can't locate the file. I tested again, this time using:

LibraryProduct(["libQt5Gui", "QtGui.framework", "QtGui"], :libqt5gui)

The error is:

[ Info: Could not locate libQt5Gui, QtGui.framework, QtGui inside ["/home/user/src/julia/builders/qt5builder/build/x86_64-apple-darwin14/uIp3iGu6/destdir/lib64", "/home/user/src/julia/builders/qt5builder/build/x86_64-apple-darwin14/uIp3iGu6/destdir/lib"]

Even though /home/user/src/julia/builders/qt5builder/build/x86_64-apple-darwin14/uIp3iGu6/destdir/lib/QtGui.framework exists.

I guess LibraryProduct doesn't support Mac frameworks? It's not that a big deal for QML.jl actually, since I don't need to load the QtGui lib from Julia, I just had defined it to make sure all basic components are built.

giordano commented 4 years ago

I guess LibraryProduct doesn't support Mac frameworks?

Yeah, most likely. Maybe worth opening a separate issue?

It's not that a big deal for QML.jl actually, since I don't need to load the QtGui lib from Julia, I just had defined it to make sure all basic components are built.

You can use a FileProduct for that, then.

staticfloat commented 4 years ago

A MacOS framework is a folder; you need to point to the dynamic library contained within that folder. BB doesn't support just listing the framework directory as a LibraryProduct; you need to give it the path to the library inside of the framework directory.

As an example, my python installation from Homebrew has /usr/local/Cellar/python/3.7.5/Frameworks/Python.framework; but that's just a folder. Within it, I can find /usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/libpython3.7.dylib, which is what BB would want to find. The fact that the libraries bundled within this framework aren't sitting directly in lib isn't that big of a problem; you just need to figure out the path to the libraries, then pass that to your LibraryProduct so that it knows where to look for the libraries (it won't recursively search the library directory for libraries on its own).

barche commented 4 years ago

Update: for the macOS build, I went to the next step and tried to link jlqml (the C++ component of QML.jl) to the BinaryBuilder-compiled Qt5, but it fails to link:

[ 15%] Linking CXX shared library lib/libjlqml.dylib
/usr/bin/cmake -E cmake_link_script CMakeFiles/jlqml.dir/link.txt --verbose=true
/opt/bin/clang++ --sysroot=/opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root -O3 -DNDEBUG -dynamiclib -Wl,-headerpad_max_install_names  -o lib/libjlqml.dylib -install_name @rpath/libjlqml.dylib CMakeFiles/jlqml.dir/application_manager.cpp.o CMakeFiles/jlqml.dir/julia_api.cpp.o CMakeFiles/jlqml.dir/julia_display.cpp.o CMakeFiles/jlqml.dir/julia_function.cpp.o CMakeFiles/jlqml.dir/julia_painteditem.cpp.o CMakeFiles/jlqml.dir/julia_signals.cpp.o CMakeFiles/jlqml.dir/listmodel.cpp.o CMakeFiles/jlqml.dir/makie_viewport.cpp.o CMakeFiles/jlqml.dir/opengl_viewport.cpp.o CMakeFiles/jlqml.dir/wrap_qml.cpp.o CMakeFiles/jlqml.dir/jlqml_autogen/mocs_compilation.cpp.o /workspace/destdir/lib/QtQuick.framework/QtQuick /workspace/destdir/lib/QtSvg.framework/QtSvg /workspace/destdir/lib/QtWidgets.framework/QtWidgets /workspace/destdir/lib/libcxxwrap_julia_stl.dylib /workspace/destdir/lib/QtQml.framework/QtQml /workspace/destdir/lib/QtNetwork.framework/QtNetwork /workspace/destdir/lib/QtGui.framework/QtGui /workspace/destdir/lib/QtCore.framework/QtCore /opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root/System/Library/Frameworks/OpenGL.framework /opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root/System/Library/Frameworks/AGL.framework /workspace/destdir/lib/libcxxwrap_julia.0.6.5.dylib /workspace/srcdir/julia/lib/libjulia.dylib -framework CoreFoundation 
clang-9: warning: argument unused during compilation: '-nostdinc++' [-Wunused-command-line-argument]
ld: can't map file, errno=19 file '/opt/x86_64-apple-darwin14/x86_64-apple-darwin14/sys-root/System/Library/Frameworks/AGL.framework' for architecture x86_64
clang-9: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [CMakeFiles/jlqml.dir/build.make:246: lib/libjlqml.dylib] Error 1
make[2]: Leaving directory '/workspace/srcdir/build'
make[1]: *** [CMakeFiles/Makefile2:73: CMakeFiles/jlqml.dir/all] Error 2
make[1]: Leaving directory '/workspace/srcdir/build'
make: *** [Makefile:130: all] Error 2

The cmake configuration command was:

cmake -DJulia_PREFIX=$Julia_PREFIX -DJlCxx_DIR=$prefix/lib/cmake/JlCxx -DQt5Core_DIR=$prefix/lib/cmake/Qt5Core -DQt5Quick_DIR=$prefix/lib/cmake/Qt5Quick -DQt5Svg_DIR=$prefix/lib/cmake/Qt5Svg -DQt5Widgets_DIR=$prefix/lib/cmake/Qt5Widgets -DCMAKE_INSTALL_PREFIX=$prefix -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER_ID=AppleClang -DCMAKE_CXX_COMPILER_VERSION=10.0.0 -DCMAKE_CXX_STANDARD_COMPUTED_DEFAULT=11 ../jlqml/

Any ideas?

giordano commented 4 years ago

I've never seen this error, have you tried already to Google the error message? I found for example:

Not sure they help though

barche commented 4 years ago

This was fixed by simply not linking to AGL. Next problem, when testing the binaries I get:

dyld: lazy symbol binding failed: Symbol not found: ___isOSVersionAtLeast
  Referenced from: /Users/user/src/build/jlqml/plugins/platforms/libqcocoa.dylib
  Expected in: flat namespace

dyld: Symbol not found: ___isOSVersionAtLeast
  Referenced from: /Users/user/src/build/jlqml/plugins/platforms/libqcocoa.dylib
  Expected in: flat namespace

This ___isOSVersionAtLeast symbol is not available in the cross-compilation environment, so it requires lazy linking (flags "-Wl,-undefined -Wl,dynamic_lookup"), but for some reason it is subsequently not found at runtime. I hit this with libjlqml.so itself as well, but there it was solved by opening the library with RTLD_GLOBAL. I don't know how to do that with libqcocoa.dylib however, since that is loaded by Qt itself :(

BTW, is there an easy way to specify binary dependencies built locally for testing? Right now I have to launch the jlqml build script, extract the Qt binaries manually in the debug shell and then tar up the destdir, and manually fix all the rpaths just to get to test it in QML.jl.

giordano commented 4 years ago

BTW, is there an easy way to specify binary dependencies built locally for testing? Right now I have to launch the jlqml build script, extract the Qt binaries manually in the debug shell and then tar up the destdir, and manually fix all the rpaths just to get to test it in QML.jl.

You can deploy the JLL package to your own repo with the --deploy flag to the build_tarballs.jl script:

            --deploy=<repo>     Deploy binaries and JLL wrapper code to a github
                                release of an autogenerated repository.  Uses
                                `github.com/JuliaBinaryWrappers/<name>_jll.jl` by
                                default, unless `<repo>` is set, in which case it
                                should be set as `<owner>/<name>_jll.jl`.  Setting
                                this option is equivalent to setting `--deploy-bin`
                                and `--deploy-jll`.
barche commented 4 years ago

Good news, to solve the unresolved symbol, all that was needed was an upgrade to the macOS SDK 10.14, and also set that as minimal version. The deploy also worked, the package is at https://github.com/barche/qt52_jll.jl, but I can't seem to specify that as a dependency in another build.jl directly? I tried this:

dependencies = [
    "libcxxwrap_julia_jll",
    PackageSpec(url="https://github.com/barche/qt52_jll.jl.git",name="qt52_jll",uuid="b32106f1-233a-535c-a12b-277d5c447d76",rev="qt52-v5.13.2+0")
]
giordano commented 4 years ago

Try removing the build number, Pkg doesn't like it. (well, this is isn't a Pkg version, so it shouldn't actually be a problem)

Edit: or maybe give the hash?

barche commented 4 years ago

I couldn't figure out any way to specify the source, so I manually hacked my local package registry to have the qt52_jll package. After that, I was able to build libjlqml, which is now here: https://github.com/barche/jlqml_jll.jl

However, after adding this, I get:

ERROR: LoadError: InitError: could not load library "/Users/user/.julia/artifacts/f98e9a38daa3fdd36e5de7030854d21d179f0497/lib/libjlqml.dylib"
dlopen(/Users/user/.julia/artifacts/f98e9a38daa3fdd36e5de7030854d21d179f0497/lib/libjlqml.dylib, 1): Library not loaded: @rpath/QtQuick
  Referenced from: /Users/user/.julia/artifacts/f98e9a38daa3fdd36e5de7030854d21d179f0497/lib/libjlqml.dylib
  Reason: image not found

Could this be because QtQuick is not declared as a LibraryProduct in the Qt5 jll?

giordano commented 4 years ago

Could this be because QtQuick is not declared as a LibraryProduct in the Qt5 jll?

That's possible, as libraries that are listed as LibraryProduct are automatically dlopened by the JLL package. Does it work if you manually dlopen QtQuick before?

barche commented 4 years ago

Good idea, and yes, that did the trick.

So the reason it was not listed as a LibraryProduct is that I never figured out how to add the framework as a LibraryProduct. The problem is that Qt leaves off the extension, and I think BinaryBuilder wants a .dylib on MacOS. The full structure (skipping headers) of e.g. QtQuick.framework is:

QtQuick.framework
├── Headers -> Versions/Current/Headers
├── QtQuick -> Versions/Current/QtQuick
├── QtQuick.prl
├── Resources -> Versions/Current/Resources
└── Versions
    ├── 5
    │   ├── QtQuick
    │   └── Resources
    │       └── Info.plist
    └── Current -> 5

I'll dig into this a little further, but I think this will end in a PR to explicitly add framework support for LibraryProduct.

giordano commented 4 years ago

I'll dig into this a little further, but I think this will end in a PR to explicitly add framework support for LibraryProduct.

That would be great, thanks.

I think this requires a fix also in Pkg: Pkg.BinaryPlatforms.parse_dl_name_version doesn't like libraries without extensions:

julia> parse_dl_name_version("QtQuick", MacOS())
ERROR: ArgumentError: Invalid dynamic library path 'QtQuick'
Stacktrace:
 [1] parse_dl_name_version(::String, ::MacOS) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/BinaryPlatforms.jl:643
 [2] top-level scope at REPL[26]:1

Workaround: manually rename all these files?

Also: @staticfloat does parse_dl_name_version need to be in Pkg.BinaryPlatforms? Doesn't seem to be used anywhere.

staticfloat commented 4 years ago

does parse_dl_name_version need to be in Pkg.BinaryPlatforms? Doesn't seem to be used anywhere.

It's used in valid_dl_path() and detect_libgfortran_version(); which is used to determine the libgfortran version for the currently-running Julia within detect_compiler_abi. So yeah, it needs to stay in Pkg.

I'll dig into this a little further, but I think this will end in a PR to explicitly add framework support for LibraryProduct.

We could make a new product (a FrameworkProduct) since I think this is going to be a little different. For one thing, we don't need to worry about it being a valid library name; and we can calculate exactly where it's going to be; we don't have to search lib and lib64 and whatnot, we can just look in $name.framework/Versions/Current/$name.

barche commented 4 years ago

We could make a new product (a FrameworkProduct) since I think this is going to be a little different. For one thing, we don't need to worry about it being a valid library name; and we can calculate exactly where it's going to be; we don't have to search lib and lib64 and whatnot, we can just look in $name.framework/Versions/Current/$name.

Would it make sense then to make LibraryProduct abstract in that case, rename the current LibraryProduct to something like SharedLibraryProduct and then have both FrameworkProduct and SharedLibraryProduct inherit from LibraryProduct?

giordano commented 4 years ago

It's used in valid_dl_path() and detect_libgfortran_version(); which is used to determine the libgfortran version for the currently-running Julia within detect_compiler_abi. So yeah, it needs to stay in Pkg.

Memo to self: always remember to open the files GitHub search finds, and look in there for other occurrences.

Would it make sense then to make LibraryProduct abstract in that case, rename the current LibraryProduct to something like SharedLibraryProduct and then have both FrameworkProduct and SharedLibraryProduct inherit from LibraryProduct?

Even though what you suggest makes sense, it'll be quite disruptive, unless you do:

LibraryProduct(x) = SharedLibraryProduct(x)

so to keep existing builders working.

My main concern about FrameworkProduct is that it would be macOS-specific, which would probably require multiple build_tarballs() calls like what we do for Git_jll, which is far less than ideal. I don't have a better alternative at the moment though.

staticfloat commented 4 years ago

For something that has frameworks, I think we will have to be content with complicated build_tarballs.jl files. :( They've clearly put a lot of work into making it as native on MacOS as possible, and I think it's reasonable for us to bend a bit to do the same.

I don't think a FrameworkProduct is fundamentally a LibraryProduct; I think having them be separate makes sense; a Framework implies more than just a shared library; it comes with data, potentially multiple versions, etc.... I think it would be a mistake to make it act just like a library product. We probably want to have a FrameworkProduct export a symbol just like a LibraryProduct, but also export a path to the framework root like a FileProduct, for instance.

barche commented 4 years ago

I did a new test using PR #698 and it seems to work on Mac. So now it's just a matter of polishing things up and getting PR #698 and #701 merged, and then QML.jl can work with a simple Pkg.add.