search

dhall-lang / dhall-haskell

Maintainable configuration files
https://dhall-lang.org/
BSD 3-Clause "New" or "Revised" License
912 stars 213 forks source link

fixing the MacOS build #2586

Closed winitzki closed 4 months ago

winitzki commented 4 months ago

The goal of this PR is to fix any problems with the build of dhall-haskell.

Current issues:

The error is in installing libsodium. 

tried: '/opt/homebrew/lib/libsodium.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64'))

This discussion https://stackoverflow.com/questions/72970363 suggests using the command: arch -x86_64 /usr/local/bin/brew install libsodium

winitzki commented 4 months ago

Currently the macos build seems to be broken. I've been trying randomly poking around to fix it but I don't really know what I'm doing in this build system. https://github.com/dhall-lang/dhall-haskell/pull/2586 Anyone has any ideas? The failure is around libsodium and has to do with macos architectures (x86 vs arm).

The error seems to be confusingly worded. The error message says that it found libsodium with arm64 architecture but it requires libsodium with x86 architecture. However, it is wrong that it should have looked for libsodium with x86 architecture, because the build is for macos-latest which is arm64. See here for the list. https://github.com/actions/runner-images?tab=readme-ov-file#available-images

It seems that macos-latest is configured to build arm64. However, I am also not able to change that into building x86.

What I have tried so far:

winitzki commented 4 months ago

The build fails immediately if the runner is set to macos-14, macos-14-large, macos-latest-large. (Although those runners are supposed to be working now. https://github.com/actions/runner-images?tab=readme-ov-file#available-images )

The previous working version was macos-13 and I'm trying it right now. However, the github settings for this repo require macOS-latest build to succeed.

winitzki commented 4 months ago

I found this page: https://docs.github.com/en/actions/using-github-hosted-runners/about-larger-runners/about-larger-runners#about-macos-larger-runners which says:

Larger runners are only available for organizations and enterprises using the GitHub Team or GitHub Enterprise Cloud plans. Probably, this is why the CI job simply fails immediately when I specified macos-14-large or other "large" runners.

So, we have a choice:

mmhat commented 4 months ago

@winitzki Thank's for your efforts; I am trying to understand why this happens and here is what I have found so far:

I suspect the reason is that GH changed the runners to arm64 only: https://github.com/actions/runner-images/issues/9741#issuecomment-2075569220 Their issue tracker has a bunch of interesting issues in this regard, involving caching issues (e.g. https://github.com/actions/runner-images/issues/9741#issuecomment-2080025306), wrong uname -m output (https://github.com/actions/runner-images/issues/9755#issuecomment-2081024845). Apparently Homebrew choked on that change as well (https://github.com/actions/runner-images/issues/9760), and there are some more interesting issues here: https://github.com/actions/runner-images/issues/9766 https://github.com/actions/runner-images/issues/9676

I also re-run an old GH action of this PR with debug logging enabled: https://github.com/dhall-lang/dhall-haskell/actions/runs/9049961867/job/25621271094 There is something weird going on: For example https://github.com/dhall-lang/dhall-haskell/actions/runs/9049961867/job/25621271094#step:3:73 indicates that we are running on arm64 architecture (which is correct), and then a few lines later (e.g. https://github.com/dhall-lang/dhall-haskell/actions/runs/9049961867/job/25621271094#step:3:117) ghcup fetches and installs an x86_64 bindist of GHC. Later, stack appears to build x86_64 objects as well (See e.g. https://github.com/dhall-lang/dhall-haskell/actions/runs/9049961867/job/25621271094#step:6:2309).

mmhat commented 4 months ago

@winitzki It might be worth trying to use the latest haskell setup action -- Which is haskell-actions/setup@v2.7.

The various "Downloading ..." debug messages (e.g. https://github.com/dhall-lang/dhall-haskell/actions/runs/9049961867/job/25621271094#step:3:465) in the GH Actions run linked previously originate from the haskell setup action: See https://github.com/haskell-actions/setup/blob/33585e1a16afa5875e124b0ebc89dd0c2f872c21/dist/index.js#L4523 in the definition and https://github.com/haskell-actions/setup/blob/33585e1a16afa5875e124b0ebc89dd0c2f872c21/src/installer.ts#L343 for the usage of the downloadTool function.

winitzki commented 4 months ago

I think we should pin the MacOS runner first in order to get working mac builds again, but #2587 should stay open and the subject should be adjusted. Thought?

In my view, the root cause is that dhall-haskell cannot build under macos-arm64 due to some dependency problems in hnix-store and/or elsewhere. The macos-arm64 build was actually never tested in dhall-haskell, and in fact there were never any macos-arm64 executables released for dhall. When github changed the macos-latest runners to arm64, all builds started failing.

To resolve this issue, I propose the following path forward:

  1. Use macos-13 instead of macos-latest (as in this PR). Change the repo settings so that macos-latest is no longer a requirement for passing the CI builds. (I don't have permissions to do this.) Mention in the README that macos-arm64 is currently unsupported.
  2. Fix the dependency problems so that macos-arm64 build becomes successful. Enable macos-13 and macos-latest builds and publish both dhall-darwin-x86 and dhall-darwin-arm64 executables. Mention in the README that macos-arm64 is now supported.
winitzki commented 4 months ago

The macos-latest build passed with haskell actions 2.7. Thank you for suggesting that!

Let me add a macos-13 build as well.

winitzki commented 4 months ago

Let me know if this is OK to merge. This will unblock other builds.

What needs to be done in order to publish both macos-x86 and macos-arm64 executables when a release is made?

mmhat commented 4 months ago

I think we should pin the MacOS runner first in order to get working mac builds again, but #2587 should stay open and the subject should be adjusted. Thought?

In my view, the root cause is that dhall-haskell cannot build under macos-arm64 due to some dependency problems in hnix-store and/or elsewhere. The macos-arm64 build was actually never tested in dhall-haskell, and in fact there were never any macos-arm64 executables released for dhall. When github changed the macos-latest runners to arm64, all builds started failing.

Indeed.

To resolve this issue, I propose the following path forward:

  1. Use macos-13 instead of macos-latest (as in this PR). Change the repo settings so that macos-latest is no longer a requirement for passing the CI builds. (I don't have permissions to do this.) Mention in the README that macos-arm64 is currently unsupported.

Would you mind adding that comment to the README?

  1. Fix the dependency problems so that macos-arm64 build becomes successful. Enable macos-13 and macos-latest builds and publish both dhall-darwin-x86 and dhall-darwin-arm64 executables. Mention in the README that macos-arm64 is now supported.

:+1:

Let me know if this is OK to merge. This will unblock other builds.

LGTM, merge it when you're done. If you include bigger changes, don't hesitate to ask for another review!

What needs to be done in order to publish both macos-x86 and macos-arm64 executables when a release is made?

I'd proceed as follows:

  1. Parameterize the objects of the workflow with runner.arch from https://docs.github.com/en/actions/learn-github-actions/contexts#runner-context. In particular https://github.com/dhall-lang/dhall-haskell/blob/e281834c9ae9b27e7e42810465c8802b66fd6488/.github/workflows/main.yml#L31, https://github.com/dhall-lang/dhall-haskell/blob/e281834c9ae9b27e7e42810465c8802b66fd6488/.github/workflows/main.yml#L47 (maybe the restore-key: too) and https://github.com/dhall-lang/dhall-haskell/blob/e281834c9ae9b27e7e42810465c8802b66fd6488/.github/workflows/main.yml#L158
  2. Also use runner.arch in the conditional executions here: https://github.com/dhall-lang/dhall-haskell/blob/e281834c9ae9b27e7e42810465c8802b66fd6488/.github/workflows/main.yml#L74, https://github.com/dhall-lang/dhall-haskell/blob/e281834c9ae9b27e7e42810465c8802b66fd6488/.github/workflows/main.yml#L99, https://github.com/dhall-lang/dhall-haskell/blob/e281834c9ae9b27e7e42810465c8802b66fd6488/.github/workflows/main.yml#L148
  3. I think it is good to get rid of https://github.com/dhall-lang/dhall-haskell/blob/e281834c9ae9b27e7e42810465c8802b66fd6488/.github/workflows/main.yml#L130 and use runner.arch in https://github.com/dhall-lang/dhall-haskell/blob/e281834c9ae9b27e7e42810465c8802b66fd6488/.github/workflows/main.yml#L137.
winitzki commented 4 months ago

@mmhat The current build with macOS-latest is an arm64 build, correct?

https://github.com/dhall-lang/dhall-haskell/actions/runs/9316129247/job/25643704112?pr=2586

Current runner version: '2.316.1'
Operating System
  macOS
  14.5
  23F79
Runner Image
  Image: macos-14-arm64
  Version: 20240526.2
  Included Software: https://github.com/actions/runner-images/blob/macos-14-arm64/20240526.2/images/macos/macos-14-arm64-Readme.md
  Image Release: [https://g](https://github.com/actions/runner-images/releases/tag/macos-14-arm64%2F20240526.2)

So, it looks like macos-arm64 build is working and passing all tests. I don't think we need to say in the readme that it is unsupported. Should we?