nasa / radbelt

radbelt: An Astropy-friendly wrapper for the AE-8/AP-8 Van Allen belt model
Other
28 stars 9 forks source link

Branch out builds for operating systems and architectures #59

Closed kokroo closed 5 months ago

kokroo commented 10 months ago

While working on Apple Silicon (https://github.com/nasa/radbelt/pull/58) and Windows builds for radbelt, it was discovered that some extra steps are needed in both cases.

For Apple Silicon, we need to set up a forked compiler while for Windows, some DLL files need to be linked. While building musllinux wheels, some dependencies are compiled from the source, which can be overcome by linking their pre-compiled versions.

To accommodate all of these steps, we can encapsulate them into reusable GitHub actions as proposed by @lpsinger

The current GitHub workflow file uses cibuildwheel without explicitly specifying combinations of operating systems and architectures. Since cibuildwheel does not expose any way to detect the current target architecture being built, there is no way to utilize custom steps for macOS, Windows, and musllinux as proposed above.

To overcome this limitation, the following can be done:

jobs:
  wheels:
    name: Build wheels
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        include:
          - {os: macos-latest, arch: x86_64, build: "*"}
          - {os: macos-latest, arch: arm64, build: "*"}
          - {os: windows-latest, arch: auto, build: "*"}
          - {os: ubuntu-latest, arch: "auto aarch64", build: "*"}
          - {os: ubuntu-latest, arch: "auto aarch64", build: "*musllinux*"}

    steps:
      - uses: actions/checkout@v3

      - name: Set up QEMU
        if: runner.os == 'Linux'
        uses: docker/setup-qemu-action@v2
        with:
          platforms: arm64

      - name: Set up macOS arm64 compiler
        if: matrix.os == ‘macos-latest’ && matrix.arch == ‘arm64’
        uses: kokroo/cross-compile-fortran-macos@v1

      - name: Set up Windows DLL linker
        if: matrix.os == windows-latest’
        uses: kokroo/fortran-dll-linker@v1

      - name: Set up musllinux pre-compiled packages
        if: matrix.os == ‘ubuntu-latest’ && matrix.build == ‘*musllinux*’
        uses: kokroo/link-precompiled-packages@v1

      - name: Build wheels
        uses: pypa/cibuildwheel@v2.14
        env:
          CIBW_ARCHS: ${{ matrix.arch }}
          CIBW_BUILD: ${{ matrix.build }}
          # Skip PyPy wheels
          # Skip 32-bit Intel wheels
          # Skip musllinux unless explicitly overridden by matrix.build
          CIBW_SKIP: '*musllinux* *pp* *_i686'

This kind of workflow will allow us to detect the OS and arch, while allowing for custom build steps in a clean fashion.

As seen here (https://pypistats.org/packages/astropy), astropy is downloaded between 15000 and 30000 times daily, with Windows and macOS users accounting for around 5% each daily, which is significant. Making radbelt available on all these platforms will benefit all these potential users.

Screenshot 2023-08-30 at 17 56 21
kokroo commented 10 months ago

@lpsinger I would love to know your thoughts about this. If you agree with it, I will open a PR for this change first, so that macOS, Windows, and musllinux wheels can be merged easily later.

lpsinger commented 10 months ago

Yes, that's what I asked for here: https://github.com/nasa/radbelt/pull/58#issuecomment-1695456255

kokroo commented 10 months ago

@lpsinger Can I open this as a separate PR?

lpsinger commented 10 months ago

Whatever you like. I think it would be simpler to do that in #58, since this is what I asked for in review of #58.

lpsinger commented 5 months ago

Fixed by #69.