search

crystal-lang / distribution-scripts

40 stars 24 forks source link

Update omnibus #98

Closed maxfierke closed 3 years ago

maxfierke commented 3 years ago

This PR updates omnibus to a version with support for running and compiling different packages on an ARM64-based Mac (i.e. the M1)

Through this, I introduced -DLLVM_ENABLE_LIBXML2=OFF (discussed also in #96) to the LLVM build, as well. On Big Sur, when compiling LLVM 10, you end up with a garbage -llibxml2.tdd in the default system libs. It doesn't seem to be needed. This is fixed in LLVM 11, but we're not there yet.

This is the first PR for compiling the Crystal omnibus package for ARM64 on LLVM 10.

maxfierke commented 3 years ago

The next step on our side would be to build an llvm-10 again using a recent XCode (12.2+, I think) as per #99 (comment)

XCode 12.2+ I think is only needed on arm64 for linking the final binary. Shouldn't matter for Intel (unless we want to implement full cross compilation from Intel to AArch64 via Omnibus). The LLVM 10 rebuild on Intel would be more for adding the AArch64 platform to the build so the Intel builds could cross-compile for AArch64.

bcardiff commented 3 years ago

A new package iteration for llvm 10.0.0 with LLVM_TARGETS_TO_BUILD="X86;AArch64" has been uploaded to https://crystal-lang.s3.amazonaws.com/llvm/llvm-10.0.0-2-darwin-x86_64.tar.gz . This was built by c63f62be4c2444d388367cf1691e5ab71455e2f7 (and I manually tweak the package iteration later 🤦 )

I am triggering a maintenance build on the compiler https://github.com/crystal-lang/crystal/commit/e38bfff6f8642ec0d0953ebe523e8329d2a2594e that we can test and accept this.

If everything goes well I might add ca523ba9d382b16fb9b1a03ca843a70bd0161f48 to this PR before merging.

FTR:

bcardiff commented 3 years ago

Ok, things need some more tweaks on paths.

I was able to get a package using llvm-10.0.0-2 in https://github.com/crystal-lang/crystal/commit/f0440a477677681bbfc0e8d2ffcdfee79b036fda and checked a 2nd generation from it in https://github.com/crystal-lang/crystal/commit/5d93d84add7643d658a30e389c057e071216d023

I smoke tested both on Catalina 10.15.7 and everything seems fine.

Next steps:

@maxfierke you will notice some overlapping changes on the makefile with your other PRs, since the pkg had the arch suffix after the omnibus update I needed to update the Makefile also, and I apply your other changes there too.

maxfierke commented 3 years ago

That all makes sense and sounds good to me. One potential snag with (c) is that we'd need to put some stuff into the omnibus setup for cross-compiling libcrystal.a if we want to compile 1.0.0 for M1, since that release pre-dates the removal of the C-based segfault handler. But such an ad-hoc build could also be done specially too, I suppose and not need to have changes committed to the omnibus setup.

Let me know if there's anything else you need from me pre-merge (or post-merge too) in terms of testing or builds! Thank you so much for all your help and time on this!

(Related: I'm looking further at whether the omnibus can be cross-compiled for arm64 from x86_64, e.g. for CI. It'll require a build of LLVM for each arch since the LLVM libs are only compiled for the LLVM host target arch and not all the targets supported by the build. I don't think it will be ready during the 1.1.0 freeze, but might make things easier in the future. Essentially, we could build LLVM for aarch64/arm64 and LLVM for x86_64, then compile Crystal targeting aarch64 via x86_64 LLVM, and then link with arm64 LLVM libs.)

bcardiff commented 3 years ago

Oh, the libcrystal.a dependency is definitely something might tempt us to not do 1.0.

bcardiff commented 3 years ago

Oh, the libcrystal.a dependency is definitely something might tempt us to not do 1.0.