Support for VMs on Apple M1

[pmatos]

Is there any plan to release VMs in Apple M1 hardware to run GitHub Actions?

[thejoebourneidentity]

Hi @pmatos, at this moment we do not have a ETA for enabling M1 support in the GitHub Actions hosted runners, but we are actively evaluating the feasibility of this and will provide updates on our roadmap when we have something to share!

[tomiiide]

@thejoebourneidentity Any update on this yet?

[paulocoutinhox]

AWS Now Offer It: https://9to5mac.com/2021/02/02/m1-mac-mini-in-the-cloud/

[QuiiBz]

AWS Now Offer It: https://9to5mac.com/2021/02/02/m1-mac-mini-in-the-cloud/

It’s Scaleway who provides M1 Macs Mini, AWS doesn’t have M1 yet - only Intel. On the other hand, I believe that AWS will have M1 in a near future.

[ioquatix]

Has anyone tried the self-hosted runner on a local M1?

[lloeki]

@ioquatix yes I have. It works but it's quite annoying in the current state of GHA:

• The runner process itself runs under Rosetta because at some point it hinges on Intel binaries.
• I have overriden Rosetta 2 by using arch(1) on each run step: arch -arch arm64 my command. It's very manual but it works perfectly to drop back into non-Rosetta land. ~Unfortunately without writing a full-blown custom action, GHA only allows a limited number of shells (I wish it would be possible to override the shell -command- altogether, not just the shell program from a fixed list)~
• The GHA UI only allows one to pick from a very limited list†, and thus I created it as macOS x86_64, which automatically adds unremovable tags macOS and x86_64 on the runner. It is kind of true since there's Rosetta available but it may not be what you want, as if you have other custom macOS runners on Intel it may still pick up non-arm64 jobs, but you may want to keep it free of this for resource management purposes, as well as not complicating the steps with more conditionals on the arch commands above.
• I added a manual arm64 tag to distinguish it.
• It is only possible to add a runner to one GH project or one GH organisation, and with the current implementation you can really only have one runner per user on a machine††. This is fairly limiting.
• Since the runner lives as a process under a regular user (which I created one specifically for), unless you take extra measures (at the -very- least chmod 700 all other users home dirs) the runner has trivial access to everything on the machine, as well as LAN access†††. This concern is mentioned by GH when adding a new runner under the synthetic warning of "don't use custom runners on public projects". This concern would be largely solved if macOS were virtualisable on ARM as it is on Intel, but currently comes short of any VM solution to achieve that. That, or dedicate the machine to be a pure GH runner that never runs for private projects, and put it in a DMZ.

† Why they don't allow a blank "custom" platform (at the cost of being on your own) is beyond me.

†† I suppose one could hack the thing to allow more but the runner CLI does not make that easy out of the box.

††† That's ultimately why I shut it down as it's on my home LAN and contains personal data on a separate account.

[lloeki]

@ioquatix that's basically what I used (just follow the registering steps of macOS x64 provided by GHA's UI):

name: Build
on:
    - push

jobs:
  try-macos-arm64:
    name: Try macOS on M1
    runs-on: [self-hosted, macOS, ARM64]
    defaults:
      run:
        shell: "/usr/bin/arch -arch arm64e /bin/bash {0}"
    steps:
        - name: Status
          run: |
            clang --version
            uname -a
        - name: Checkout
          uses: actions/checkout@v2
        - name: Set Ruby metadata
          id: ruby
          run: |
              ruby -e 'puts RUBY_PLATFORM' | tee ruby_platform
              echo "::set-output name=RUBY_PLATFORM::$(cat ruby_platform)"
              ruby -e 'puts Gem.platforms.last.to_s' | tee gem_platform
              echo "::set-output name=GEM_PLATFORM::$(cat gem_platform)"

(A note for Ruby users, please do yourself a favour and install a Ruby >= 2.7.2, as Apple's Ruby 2.6 seems to miss some important stuff)

[maxim-lobanov]

Related issue: https://github.com/actions/runner/issues/805

[lloeki]

@QuiiBz There's MacStadium as well: https://www.macstadium.com/configure?p=minig5&c=VGS2

Anyway even if there were M1 machines on AWS the cost is absolutely ludicrous. I don't see anyone using that unless they're heavily invested (a.k.a trapped with no other option) in AWS as a management platform and need it at a huge enough scale to offset the cost.

[alirexa]

@thejoebourneidentity do we have an ETA on this yet?!

[mikehardy]

I think the macstadium founder wrote an interesting blog about the changes in macos licensing for big sur, specifically around virtual provision of Mac hardware. Previously it was unspecified but now it's less ambiguous and I believe specifies a minimum time slice of 24hrs. If I understand that correctly I believe it means macos big sur and higher is not really usable in a cloud CI provider environment. Coincides with github actions still being on older macos with older xcode 🤔. May be an intractable problem, or an expensive one at least (clients need to fund the larger time slices / less efficient statistical multiplexing of build hardware). Would be nice to have some official explanation of the delay on all aspects of the macos infrastructure here

[adamcstephens]

Coincides with github actions still being on older macos with older xcode 🤔.

Actions supports the macos-11 target already, they just haven't pointed macos-latest at the Big Sur images. https://github.com/actions/virtual-environments/blob/main/images/macos/macos-11-Readme.md

[lloeki]

Previously it was unspecified but now it's less ambiguous and I believe specifies a minimum time slice of 24hrs.

(caveat: IANAL) There's a lot of legal verbiage (in Section 3 "Leasing for Permitted Developer Services": "leasing", "subleasing", "lessee", "end user lessee", "permitted developer services"...) to navigate through and my last read of it was that the ones renting the hardware (the lessor, e.g MacStadium) must lease it for 24hrs minimum to their customers, but that the text also allows to rent it like so to e.g. GitHub, and being the customer (or "lessee"), is then able to bill CI by the minute (that would be for "permitted developer services"). What's kind of obscure to me is that it could also count as "subleasing" from GitHub to the CI users, but then it would prevent any sensible CI billing arrangement as we know it.

I'm not sure how the Scaleway model fits in that though, as they seem to both host M1 hardware and bill by the hour (so there's no lessor/lessee arrangement).

EDIT: Ah, now I know, there's an infobox at the bottom when you try to create one saying "As required by Apple License, you must keep this instance at least 24 hours. You will be able to delete it only after 24 hours.".

[mikehardy]

I'm guessing people that are lawyers are the reason macos-latest is not macos-11 and the rollout was halted, perhaps while it was still small enough to "eat" the problem from a provisioning/cost perpective while ambiguities are ironed out. Hopefully it's obvious I'm just speculating though. I was just trying to provide some context (even if speculative) since there hasn't been any official information and simple requests were getting thumbs-ed down

More than anything, just some news, plan would be nice to have as mentioned by the comment that generated the notification which prompted my comment.

[ShadowJonathan]

I'm extremely puzzled why Apple would be specialising and enforcing such a licensing time window of their operating system in the first place, are they trying to disincentive developers to use cloud-based methods or something? And/or only keep using local developer machines? (I wouldn't be surprised if that was the end goal, considering Apple)

[mikehardy]

They sell hardware, in general, and want to incentivize that. This is probably not the place to speculate on their motives (or at least, I hope my notifications don't fill up with that, so I can leave them enabled on this issue). Speculation on GitHub actually providing M1 services is probably far enough afield ;-)

[ShadowJonathan]

I didn't mean to start speculation, sorry, the thought just slipped while reading this thread.

I do hope that github could be providing such a solution in the near future though, i'll leave it at that :)

[apps4everyone]

@thejoebourneidentity Any update on this? maybe with macOS-12?

[kenmorse]

Kind of wondering if GitHub is waiting to see what happens with Xcode Cloud...

[chadbrewbaker]

Why not use @sickcodes Docker-OS X under the stock GitHub Actions Ubuntu instance? Apple has never complained- XNU and LLVM are open source. What Apple doesn’t want is people loading non-headless QEMU.

[catthehacker]

@chadbrewbaker because the request is about specific CPU configuration which you cannot achieve with a virtual machine (macOS in KVM on ArchLinux in Docker). If you need to run just macOS, you can use macos-11 or macos-10.15.

[iMonZ]

Anything new here? This is actually pretty important!

[miketimofeev]

@iMonZ we are working on it, but can not provide ETA for now

[lloeki]

@miketimofeev thanks for the info. To manage our expectations, would you be able to provide an order of magnitude (not even a ballpark), like are we talking about a week, month, quarter, year here? I promise we won't hold you folks accountable for it ;)

[Steve-Glass]

@miketimofeev thanks for the info. To manage our expectations, would you be able to provide an order of magnitude (not even a ballpark), like are we talking about a week, month, quarter, year here? I promise we won't hold you folks accountable for it ;)

👋 @lloeki - We are still in evaluation phase for M1, but are taking the next step by developing plans for a preview in 2022.

[sobrinho]

Looking forward for this

[spuder]

Temporary workaround would be to setup a self-hosted github runner https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners

[ghost]

Any updates on this?

[lloeki]

As a workaround step I'm cross-compiling builds for arm64 on x86_64 runners for some projects by adding -arch arm64 to CFLAGS, CXXFLAGS, and LDFLAGS (or its cmake equivalent -DCMAKE_OSX_ARCHITECTURES=arm64, see e.g libddwaf) but others are trickier (e.g libv8-node, because of this issue).

Of course even for those that do work there's no way to run things (e.g run unit tests or whatever). It's already quite helpful to have builds and kind of hinge on the x86_64 tests to pass.

[geoffrey4444]

I hope you'll be able to add support for GItHub Actions for Apple Silicon...is there any update?

[blaadje]

+1

[jjangga0214]

@thejoebourneidentity

Hi.

we are actively evaluating the feasibility of this

I've stayed tuned and a year has passed. Many (especially open source) projects relying on GitHub Actions to build and release are in stuck. So end users have had to depend on Rosetta2 for quite a long time..

I hope GitHub just let us know if it's seriously planning provision of Apple Silicon in near future or not. It'd be helpful for people to decide whether to move to another CI.

Just waiting without any information is too painful.

Thanks.

[remram44]

Which other CI provider supports M1?

[mikehardy]

It should surprise no one that at the same time Apple clarified their stance on what it means to do "short term rentals" of mac machines (that is, use them for things like CI/CD) such that they must be in 24-hour increments (and thus not really usable for non-enterprise CI, like open source use of Actions here) they launched :drum: :drum: :drum: https://developer.apple.com/documentation/Xcode/About-Continuous-Integration-and-Delivery-with-Xcode-Cloud

[mikehardy]

Of note is that Xcode is a cross-compiler, I don't believe there is an issue using intel-hosted macOS to compile for arm64/apple silicon?

[phraemer]

With all due respect to everyone explaining reasons why you might not need it are missing the point of the question. Those of us who need or want M1, A) don't want to have to explain all the details of why we do, and B) simply want clarity from GitHub.

[lloeki]

I don't believe there is an issue using intel-hosted macOS to compile for arm64/apple silicon?

Some projects are easy to cross-compile (and I do that for those) others are not, if not impossible due to complex build systems.

In case you can then build you can still not test the resulting binaries, which matters a lot for some projects.

[ghost]

Which other CI provider supports M1?

CircleCI may have mentioned that support is coming (I am not affiliated with CircleCI or any other CI/CD provider and don't know anything other than what can be read at these links):

• https://github.com/ethereum/solidity/issues/12291#issuecomment-974851327
• https://circleci.canny.io/cloud-feature-requests/p/support-new-m1-arm-based-macs

Some other links that might suggest possible M1 CI support at some point

• https://www.bitrise.io/m1-preregister?utm_source=twitter&utm_medium=web&utm_campaign=m1-prereg
• https://www.scaleway.com/en/hello-m1/
• A discussion regarding Azure: https://github.com/microsoft/azure-pipelines-agent/issues/3184

[mikehardy]

I'm with you, for what it's worth. Would love native M1 builds. Just trying to post possible partial solutions for the cases where those might help. Similar to the person that kicked off this last round of comments: more than anything just hearing a plan (or "there is no plan") would be cool. Cheers

[Platin21]

Is this still happening or dead? I can't really use the Xcode stuff because it doesn't support the language I use and I also don't see myself buying into there build system which I don't want and also don't need.

[TeddyAlbina]

Just install windows or linux in another box

[jovany-wang]

Is there any update?

[G-kodes]

Is there any update? Even issues currently being grappled with?

[hossein-bakhtiari-revolut]

Is there any update? Even issues currently being grappled with?

I don't know why It's being down voted

[panekj]

I don't know why It's being down voted

Because comments like the one you quoted are frowned upon as they generate useless notifications for people subscribed to this issue. This is a good explanation why you should not do that

[fkorotkov]

Just for the info of GitHub folks. We've just open sourced virtualization toolkit for Apple Silicon called Tart which can potentially power M1 infra for actions.

[garyo]

This issue has grown so long it's hard to follow. We need M1 github actions for our CI (cross-compile is not sufficient). I understand (?) that github's hosted runners are not yet supporting M1, but can we use our own (self-hosted) M1 Macs as runners yet?

[BastianZim]

Not yet: https://github.com/actions/runner/issues/805

[gaborcsardi]

@garyo you can use self-hosted M1 runners, they work fine, I have been using them for a while. You can set them up like this: https://github.com/actions/virtual-environments/issues/2187#issuecomment-790493882

[mikehardy]

@fkorotkov very interesting! I just read up on that site

Just for the info of GitHub folks. We've just open sourced virtualization toolkit for Apple Silicon called Tart which can potentially power M1 infra for actions.

I could not find in the documentation or issue search whether nested virt is supported. Do you know if Tart does nested virt?