search

JuliaInterop / libcxxwrap-julia

C++ library for backing CxxWrap.jl
Other
84 stars 43 forks source link

Staticfloat sf/docker ci #5

Closed barche closed 6 years ago

barche commented 6 years ago

Experiment with BinaryBuilder based on the work in PR #4

barche commented 6 years ago

@staticfloat Just tried downloading the correct Julia binary, but now I get a link error: https://travis-ci.org/JuliaInterop/libcxxwrap-julia/builds/393688417#L920

Any ideas? I intend to add the other downloads depending on the $target value, is that approach viable? Also, FindJulia now works without running Julia, so that should remove the obstacles for cross-compilation, I hope.

staticfloat commented 6 years ago

Just tried downloading the correct Julia binary, but now I get a link error:

Congratulations! You found a subtle bug! We build Julia x86 on Debian 8, which ships a default glibc of version 2.19, but the builtin glibc of the BinaryBuilder environment is 2.12. These two should match, so I will be updating the BinaryBuilder i686-linux-gnu shard to be using 2.19, as that's the minimum we're requiring for Julia anyway. This will, unfortunately, require a new BinaryBuilder release to work. I suggest you try other architectures and see if there are similar mismatches for any other architectures you care about; you're the first person to test things in this particular way. :)

I intend to add the other downloads depending on the $target value, is that approach viable

Yes, that seems perfectly reasonable.

staticfloat commented 6 years ago

The change to julia-docker is given here: https://github.com/staticfloat/julia-docker/commit/42d216b7a68fca6385b11631e0d2f3044fa316c6 I'm building new binarybuilder shards, but it may be a day or two before they're integrated into BinaryBuilder.

barche commented 6 years ago

Great, thanks! I'll continue testing the other platforms. Two more questions:

staticfloat commented 6 years ago

Is it sufficient to have a Julia-v0.6 and a v0.7 version, or would it be needed to build against each minor version separately?

It should be sufficient to have one build per major version.

Also, is there support in BinaryBuilder for this, or should I parametrise the build script and call build_tarballs for each Julia version?

The latter.

What are the Julia binaries for the musl Linuces

Those are for e.g. alpine linux. The official Julia binaries are only those shown here: https://julialang.org/downloads, and for stable versions of Julia that's all you're going to be able to get. While versions of Julia do kinda-sorta exist for the more exotic architectures (ppc64le, armv7l, aarch64) they're not tested nearly as well as the x86_64 and i686 architectures, and so often fall into disrepair. It's something that will become more stable as Julia gains more adoption and more people try to run Julia on e.g. their ARMv8 boards or similar.

barche commented 6 years ago

OK, additional question for macOS: is there a way to get 7zip or some other means to extract the .dmg? See https://travis-ci.org/JuliaInterop/libcxxwrap-julia/builds/394006859#L1231

staticfloat commented 6 years ago

Yeah, unfortunately, it's kind of a pain to extract the .dmg file. The best way is to use apk (which is the alpine package manager, which is what you can use to install further tools inside of the build system. Just like everything else, those tools go away at the end of the build session) to get it. So I just tested that apk add p7zip and then 7z x Julia.dmg works. You then have to dig around to get the actual julia prefix: the path to libjulia.dylib is Julia-0.6.3/Julia-0.6.app/Contents/Resources/julia/lib/ in my case.

barche commented 6 years ago

OK, unpacking works, but now I can't find the library, even though I am giving cmake the exact filename and directory: https://github.com/JuliaInterop/libcxxwrap-julia/blob/staticfloat-sf/docker_ci/FindJulia.cmake#L109

Error here: https://travis-ci.org/JuliaInterop/libcxxwrap-julia/jobs/394412495#L1337

barche commented 6 years ago

@staticfloat Finding the library works now, using the toolchain file at /opt/$target/$target.toolchain. Is that file present for all platforms?

Unfortunately, I hit an incompatibility again, error here: https://travis-ci.org/JuliaInterop/libcxxwrap-julia/jobs/394582556#L892

staticfloat commented 6 years ago

Is that file present for all platforms?

Yes.

Unfortunately, I hit an incompatibility again

Incredibly, it looks like p7zip doesn't support symlinks from .dmg files. It is just creating a text file named libjulia.dylib that contains the text content libjulia.0.6.3.dylib. You either need to link directly against libjulia.0.6.3.dylib, or manually recreate the symlinks.

Eventually, I hope to provide .tar.gz downloads of Julia for all platforms (instead of .dmg files and .exe files) but until then, you'll have to work around these issues a bit.

barche commented 6 years ago

Incredibly, it looks like p7zip doesn't support symlinks from .dmg files.

Doh, I even saw that before and then forgot about it again. Working fine now on 0.6 for Mac, win32, win64 and linux64. For 0.7, I get an error on the dmg extraction: https://travis-ci.org/JuliaInterop/libcxxwrap-julia/jobs/394740917#L2888

It seems the file doesn't play nice with 7zip, I get the same error on my local linux machine as well, even though the file mounts normally on my mac.

Also, trying ARM 64 bit gave me an error about C++11, but I haven't investigated that yet: https://travis-ci.org/JuliaInterop/libcxxwrap-julia/jobs/394735713#L1315

Anyway, thanks for all the help! Are you coming to JuliaCon? I'll have to buy you a beer ;)

staticfloat commented 6 years ago

I get the same error on my local linux machine as well, even though the file mounts normally on my mac.

Bizarre. Likely the true solution is to instead be using .tar.gz files for this. ;)

Also, trying ARM 64 bit gave me an error about C++11, but I haven't investigated that yet:

You'll have to look at the compiler invocation and see why it fails. I can't tell from here. :)

Are you coming to JuliaCon? I'll have to buy you a beer ;)

Yep! I'll be there. Speaking about BinaryBuilder.jl of course.

barche commented 6 years ago

Manually merged.