Arm64 Support In Dockerfile

[odidev]

1

Added the support for arm64 in Dockerfile. As GHC official binaries are not yet released for arm64, I am building the GHC binaries from its source.

[odidev]

Ping @psftw could you please share your thoughts on this.

[odidev]

@psftw Kindly have a look on this PR

[psftw]

Thanks for your work on this @odidev. Unfortunately right now I don't have the time to see this through, and merging/publishing prematurely would be a mistake. This PR only addresses 8.8, which in the past I had experimented with and ran in to usability problems (many of which may be resolved now). I'm not sure of the current 8.8/8.10 status for GHC (+ tooling/ecosystem) on ARM, but I'd like to do significant testing and research to make sure we are not releasing anything we shouldn't be. I'm pretty sure a number of caveats would need to be documented for users in any case. I'd also like to have a sustainable smoke-testing process that can integrate with (or replace) the existing github actions. It is possible to temporarily spin up an ARM server and use it as a github action runner, but this is not ideal.

[GrahamCampbell]

I think this needs doing for 8.10 too?

[odidev]

Hi,

Added the support for arm64 in Dockerfile for 8.8 and 8.10 8.8.4 source for arm64 is not available so i am using 8.8.3 for arm64. GHC official support for arm64 is only available for 8.8 stretch. I have cross compiled the 8.8 stretch source in buster and it is working. GHC official support for arm64 is not available for 8.10 stretch. Hence I have only added the support for ghc 8.10 buster.

The stack consist of binary whereas binary of ghc is unavailable so i cannot verify the tar as these tar consists of the source for arm64.

[odidev]

@GrahamCampbell can you please review and let me know if any changes required?

[GrahamCampbell]

I think the best approach now would be to only add ARM support for GHC 8.10.3, since there is official support now. There are a lot of issues with ARM on 8.8, even if it does "build". The 8.8 series should be left alone.

[GrahamCampbell]

As far as I'm aware, GHC 8.10.3 is the first version of GHC where ARM64 support could actually be described as "mostly working" on Linux: https://downloads.haskell.org/~ghc/8.10.3/ghc-8.10.3-aarch64-deb10-linux.tar.xz.

[odidev]

I have amended the changes as per the given suggestions.

Now this PR only consists of the changes for 8.10 buster, here 8.10.3 ghc version is used for arm64 development and the latest version of stack available for arm64 is 2.1.3. So I have updated the stack version for arm64.

Can you please review the latest commits and let me know if any further changes is required

[GrahamCampbell]

Not up to me at this point. Up to one of the maintainers with write permissions.

[GrahamCampbell]

Likely they will say wait till the debian builds are ready, then change the code back: https://github.com/haskell/docker-haskell/issues/31#issuecomment-751467630.

[psftw]

I saw there was a GHC 8.10.4 released yesterday, so I made a few tweaks to this PR and built it on an AWS Graviton system. While I was able to get it "working", there are still critical issues with releasing an aarch64 image. Without cabal-install (and ideally Stack as well) I don't think we should ship. I would rather the issues with packaging get resolved upstream instead of hacking something here. The line in your PR to install the cabal-install package brings in another set of packages including an older GHC from upstream Debian repositories which is not what we want. Similarly, including an old version of Stack because they no longer provide binaries for aarch64 is also not going to fly.

[hasufell]

ghcup supports arm now and I also have an unofficial stack binary: https://downloads.haskell.org/ghcup/unofficial-bindists/stack/2.7.3/stack-2.7.3-linux-armv7 (not distributed... these were built in an attempt to migrate stack CI to GHC gitlab, which has first-class support for ARM)

See:

• https://github.com/haskell/docker-haskell/pull/47
• https://github.com/commercialhaskell/stack/issues/5619
• https://github.com/commercialhaskell/stack/issues/5610

[Bodigrim]

@psftw any chance to move this forward? Arm docker images for GHC would be a great help for package maintainers.

[AlistairB]

Thanks for the ping on this @Bodigrim . FYI @psftw has stepped down from maintaining this project after many years of service.

I believe the major blockers at this point are:

• stack official ARM releases. It seems that this existed for 2.7.1, but not for 2.7.3. I would be inclined to only include an official stack release.
• CI that can build the images on ARM based CPUs. Actually it seems that travis might support this https://docs.travis-ci.com/user/multi-cpu-architectures/#multi-cpu-availability.

I'll see if I can get something fully working in travis CI with stack 2.7.1 for now. If that works then perhaps we can bug the stack maintainers (or offer a hand) :)

[AlistairB]

Hmm dang, seems that the haskell github org is out of travis ci credits.

[Bodigrim]

Yeah, Travis is no longer an option for OSS maintainers. I have successful experience with ARM machines provided by drone.io, see https://github.com/haskell/bytestring/blob/master/.drone.yml and https://github.com/haskell/text/blob/master/.drone.yml

Even a cabal-only docker image would be very helpful.

[AlistairB]

Ah drone looks nice thanks I'll try that.

Yeah, ideally we get stack too, but if it is not likely to be possible anytime soon I think it can be omitted (or just provide stack 2.7.1 for now).

[AlistairB]

Although, I think someone who is a haskell org admin would need to follow https://docs.drone.io/server/provider/github/ or apply the setup bytestring has to docker-haskell? Do drone host all the 'server' and 'runner' agents for us?

[hasufell]

Arm docker images for GHC would be a great help for package maintainers.

I've been working on ghcup docker images, which are more flexible and lightweight than haskell/docker and would use the GHC CI infrastructure to build. But I haven't gotten around it yet.

At any rate, it's not hard to install ghcup in a stock arm64 image.

[Bodigrim]

@AlistairB I set up drone.io for bytestring and text myself, so you should be able to do the same for docker-haskell as soon as you have enough rights for the repo. CC @chessai Alternatively you can try emulated GitHub actions environment, e. g., https://github.com/haskell/bytestring/blob/e682cf7ad44c8299aed23e649472d8ab559109d4/.github/workflows/ci.yml#L69-L90

@hasufell true, but installing everything from the scratch takes too long for a low-latency CI, which is my use case.

[chessai]

@AlistairB I set up drone.io for bytestring and text myself, so you should be able to do the same for docker-haskell as soon as you have enough rights for the repo. CC @chessai Alternatively you can try emulated GitHub actions environment, e. g., https://github.com/haskell/bytestring/blob/e682cf7ad44c8299aed23e649472d8ab559109d4/.github/workflows/ci.yml#L69-L90

@hasufell true, but installing everything from the scratch takes too long for a low-latency CI, which is my use case.

@Bodigrim, @AlistairB already has admin access to this repo (docker-haskell). Did you mean another repo?

[Bodigrim]

Ah, good, then he should be able to set up drone.io for docker-haskell himself.

[AlistairB]

Ah hmm, ok let me have another go. The instructions seemed to suggest it needs to be done at an org level.

[AlistairB]

So I've been playing with the emulated approach. The good news I think I have a good pattern now to make the Dockerfiles processor architecture agnostic in that PR. Unfortunately the emulated approach is not working as uname -m (and dpkg-architecture --query DEB_BUILD_GNU_CPU) apparently returns the processor architecture of the real host on the CI agent, not that of the emulated ARM64 docker container.

@Bodigrim Re drone, am I correct in thinking I would need to run 'servers' and/or 'runners' myself somewhere? The instructions ask me to set up a GitHub OAuth App.

And I would need to host some http://drone.myserver/login or something?

I don't really want to host anything myself (if I am understanding this all correctly) and if I did I would probably set up a self hosted github CI agent.

I think if there are no viable non-self hosted options, it is actually not the worst to just have no CI verifying ARM in this repo. As when new releases are PRed against https://github.com/docker-library/official-images as part of the release process their CI should check everything.

[Bodigrim]

@AlistairB Drone provides both servers and runners. I remember that GitHub integration was not documented well, but I managed to setup it for bytestring and text - just do not remember how :)

Dunno, try adding .drone.yml to a fork first and see whether you can integrate it?

[AlistairB]

Ok thanks that is good to know. I will figure this out! 😅

[AlistairB]

Just an update:

• So it turns out there is a manual approval process for cloud.drone.io. I am waiting for this, then I think I am good to test the updated images on ARM.
• I have pinged the stack people on the HF slack about 2.7.3 missing aarch64 binaries and whether stack intends to produce aarch64 binaries in the future. I'd like to at least get a response from the stack people before releasing something with stack 2.7.1 (or with no stack). Another option is unofficial stack binaries, but I really think the official images should not include anything unofficial.

[AlistairB]

• Still waiting for drone.io account.. Another option is try to use haskell gitlab ci, like haskell language server, but that looks really complex to set up so would rather avoid.
• I pinged the stack people and someone put their hand up to automate and add 2.7.3 ARM64 binaries. However, I have investigated and it seems the official images people are happy to have old versions of binaries when newer versions have issues. I am happy to release with 2.7.1 (at least while I see no actual movement to on the 2.7.3 issue) and https://github.com/haskell/docker-haskell/pull/53 has this fallback logic.
• https://github.com/haskell/docker-haskell/pull/53 is ready to go minus CI testing ARM64 on drone.io . Feedback welcome.

Going to close this PR and continue with the other one. Thanks @odidev this was useful to show how it is done!