ARM CI support for haskell.nix

[angerman]

Name, email, company, job title

Moritz Angermann moritz.angermann@gmail.com, Tools Team Lead

Project Title and description

haskell.nix is a very flexible solution to build haskell project with nix. This project was conceived to allow different teams using different package setups (either cabal or stack) to build their projects seamlessly in nix, while also providing all the infrastructure to cross compile projects. While we've been using IOHK's x86_64 CI infrastructure to build and test haskell.nix, and thus also provide pre-built compilers for anyone using haskell.nix, we do not have any AArch64 machines in our CI setup and thus can not pre-build compilers for, or test them on AArch64.

I would also like to improve GHC on AArch64 as a secondary concern. This includes better code generation, and debugging and fixing memory-ordering issues in GHC's Run Time System.

Which members of the community would benefit from your work?

Everyone who uses nix and haskell, and specifically everyone who uses those on arm.

Is the code that you’re going to run 100% open source? If so, what is the URL or URLs where it is located?

haskell.nix is fully open source. So is GHC.

What infrastructure (computing resources and network access) do you need? (see: https://www.packet.net/bare-metal/)?

I envision this to be a c2.large.arm instance for the haskell.nix CI on a continuous basis. And a second c2.large.arm on a temporary basis to investigate and fix GHC issues. The systems should be separate so that there is no way that work on the second machine could in any way taint or interfere with the build products produced by the first machine, as those are supposed to be consumed by others as well. I would also provide others with ssh access to the secondary machine upon request to work on GHC on AArch64.

Describe the continuous integration (CI) system in use or desired for this project.

Haskell.nix is use heavily in a nix based CI setup. I would initially just ask that this would be added to our existing CI infrastructure and scoped to only haskell.nix builds.

Please state your contributions to the open source community and any other relevant initiatives

My first commit to GHC has been in 2014, and it was about making it build for arm. I have since contributed an aarch32 and aarch64 in-memory linker to GHC for ELF and Mach-O. Although I must admit the aarch32 in memory linker is not very well tested. I have written an alternative GHC Code Generator targetting LLVM BitCode (1, 2, 3, 4), that did not make it into GHC yet. And did start haskell.nix.

[rhenwood-arm]

Hi @angerman,

I've been in touch with the Haskell upstream community. It looks like there are still usability problems that are prohibiting a formal release for AArch64: https://github.com/haskell/docker-haskell/pull/21 . I would like to see the Haskell community to be satisfied with the quality of the AArch64 release before downstream consumers (like Haskell.Nix?) become involved.

How does your proposal relate to the broader Haskell communities' work on AArch64?

[angerman]

@rhenwood-arm right, there is an aarch64 machine that is used in CI to help build GHCs. As far as I am involved, I am writing a native code generator for aarch64 for GHC (see https://gitlab.haskell.org/ghc/ghc/-/merge_requests/3641) in my spare time.

I know that our current cross compilers we can build for GHC can produce arm and aarch64 binaries. This is at this point almost trivial with haskell.nix (https://github.com/input-output-hk/tools/blob/master/arm-test/default.nix). I however do not have powerful aarch64 equipment (the most powerful probably being a S922X, RK3399 and RPi4's braodcom chips), to make this a pleasant experience.

We do know there are concurrency bugs in the GHC RTS, which still need to be ironed out and there are open Merge Requests that used ThreadSanatiser to find some issues. As for haskell.nix, this will allow us to see how early things break (most likely they won't break during compile time), but try to shake out test failures as well. The idea being to basically build a lot of haskell software for aarch64 and run all the test-suites associated with those libraries.

As for my expertise: I'm not only involved in the GHC Native Code Gen (the primary motivation being compile time speed as the LLVM backend can be quite slow), but also wrote large parts of GHC's static in-memory linker for aarch64.

I have also for a while been providing pre-built aarch64 cross compilers for haskell (http://hackage.mobilehaskell.org/), and spent a lot of time massaging GHCs new build system (hadrian) into a form that it can produce relocatable binary distributions.

[rhenwood-arm]

@angerman : thanks for this detail - it helps resolve the concerns I had from my outside view.

[vielmetti]

Project has been approved, invitations sent and infrastructure has been set up.