search

actions / runner-images

GitHub Actions runner images
MIT License
9.17k stars 2.84k forks source link

GitHub Action: New M1 runner available to all plans, including open source 🚀 #9254

Closed Steve-Glass closed 2 months ago

Steve-Glass commented 3 months ago

Breaking changes

The new M1 runner is available for all plans, free in public repositories, and eligible to consume included free plan minutes in private repositories. The new runner operates exclusively on macOS 14 and to use it, simply update the runs-on key in your YAML workflow file to macos-14.

Change log

Target date

Available as of 1/30/2023

The motivation for the changes

N/A

Possible impact

N/A

Platforms affected

Runner images affected

Mitigation ways

N/A

fwcd commented 3 months ago

There's something strange with the Xcode installation in these runners, running xcrun --show-sdk-version outputs

xcodebuild: error: SDK "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk" cannot be located.
xcrun: error: unable to lookup item 'SDKVersion' in SDK '/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk'

See e.g. https://github.com/fwcd/m1xxx/pull/63

I can't reproduce this locally, even with xcode-select -s /Library/Developer/CommandLineTools.

darthmaim commented 3 months ago

The new macos-14 runners contain a link at the top of the log to included software:

Included Software: https://github.com/actions/runner-images/blob/macOS-14/20240116.1/images/macos/macos-14-arm64-Readme.md

This link is a 404 and should instead link to https://github.com/actions/runner-images/blob/main/images/macos/macos-14-arm64-Readme.md

Vyazovoy commented 3 months ago

@darena-patrick, looks like this ticket says exactly that. If you switch to macos-14 your jobs will be running on the M1 (aka arm64) runner.

darena-patrick commented 3 months ago

@darena-patrick, looks like this ticket says exactly that. If you switch to macos-14 your jobs will be running on the M1 (aka arm64) runner.

yea, I deleted my comment once I realized that :)

darena-patrick commented 3 months ago

@Steve-Glass any plans to support Docker on these images? Otherwise, any word on when linux/arm64 will be available?

darthmaim commented 3 months ago

@Steve-Glass any plans to support Docker on these images? Otherwise, any word on when linux/arm64 will be available?

No docker for the M1 images: https://docs.github.com/en/actions/using-github-hosted-runners/about-larger-runners/about-larger-runners#limitations-for-macos-larger-runners:

Due to a limitation of Apple's Virtualization Framework, which our hypervisor uses, nested-virtualization is not supported by arm64 runners.

And for linux arm images there is this open (but locked) issue: https://github.com/actions/runner-images/issues/5631

mikhailkoliada commented 3 months ago

The new macos-14 runners contain a link at the top of the log to included software:

Included Software: https://github.com/actions/runner-images/blob/macOS-14/20240116.1/images/macos/macos-14-arm64-Readme.md

This link is a 404 and should instead link to https://github.com/actions/runner-images/blob/main/images/macos/macos-14-arm64-Readme.md

This is expected as no full public releases happened yet, it should be fixed with the next release :)

Korijn commented 3 months ago

When is this coming to Azure Pipelines?

douglascamata commented 3 months ago

@Steve-Glass there's a huge breaking change here: the M1 processor does not support nested virtualization. This will break every workflow out there using Colima/Lima or any other VM-based solution for running Docker in macOS.

Neither QEMU nor Virtualization.framework can be used to start VMs inside the runners, because they are already VMs.

douglascamata commented 3 months ago

QEMU will report {"level":"debug","msg":"qemu[stderr]: qemu-system-aarch64: Error: HV_UNSUPPORTED","time":"2024-02-01T19:34:57Z"}

Trying to use Virtualization.framework will report {"level":"fatal","msg":"Error Domain=VZErrorDomain Code=2 Description=\"Invalid virtual machine configuration. Virtualization is not available on this hardware.\" UserInfo={\n NSLocalizedFailure = \"Invalid virtual machine configuration.\";\n NSLocalizedFailureReason = \"Virtualization is not available on this hardware.\";\n}","time":"2024-02-01T19:36:49Z"}

Vyazovoy commented 3 months ago

@douglascamata, I'm not sure that it should be called a breaking change since it was explicitly mentioned as a limitation.

douglascamata commented 3 months ago

@Vyazovoy being a documented change doesn’t change the fact that it is a breaking change. They are two different things.

cgrindel commented 3 months ago

I would like to upvote @fwcd comment. I too am seeing an error running xcrun --show-sdk-version.

Luthaf commented 3 months ago

Is the GPU available on these runners? Trying to use them with PyTorch and the MPS (i.e. Metal) backend result in error like this (See https://github.com/lab-cosmo/metatensor/actions/runs/7847356344/job/21416128283)

RuntimeError: MPS backend out of memory (MPS allocated: 0 bytes, other allocations: 0 bytes, max allowed: 7.93 GB). Tried to allocate 256 bytes on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).

Setting PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 in the env changes the error message to (see https://github.com/lab-cosmo/metatensor/actions/runs/7847470210/job/21416465675)

RuntimeError: MPS backend out of memory (MPS allocated: 0 bytes, other allocations: 0 bytes). Tried to allocate 256 bytes on shared pool.

There are other reports of the same issue on the PyTorch forums: https://discuss.pytorch.org/t/mps-back-end-out-of-memory-on-github-action/189773

wtdcode commented 3 months ago

"isb" instructions are recognized as EXC_BAD_INSTRUCTION in the new M1 runners.

janosh commented 2 months ago

@Luthaf unfortunately, the new macos-14 don't have access to MPS hardware, see https://github.com/pytorch/pytorch/issues/111449#issuecomment-1767613883.

amitdo commented 2 months ago

@Steve-Glass,

Target date

Available as of 1/30/2023

You mean 1/30/2024

ssbarnea commented 2 months ago

Apparently the macos-14 also has support to run intel code disabled, qemu reports:

Error: qemu exited unexpectedly with exit code -1, stderr: qemu-system-aarch64: -accel hvf: Error: HV_UNSUPPORTED

This a big bummer for someone wanting to build a dual architecture container for example. Doing that on an intel bacsed linux machine is like 5x slower than doing it on a macos arm based machine.

If I understand correctly, none of the M1 macos runners available from github do support any container engine (docker or podman) as they do not allow the required linux VM to be started.

That is a huge issue, especially if you combine it with the fact that there is no arm linux runner yet.

mikhailkoliada commented 2 months ago

@ssbarnea it is not a GitHub restriction, Apple does not support nested virt on M1 CPU gen entirely

qwqcode commented 2 months ago

Unfortunately, the current macos-14 runner which ARM architecture cannot run Docker.

Although it can execute builds on native ARM, this essentially says goodbye to the Docker ecosystem system. 😢

slonopotamus commented 2 months ago

What's the point using mac runners for Docker tasks if you can use Linux runners?

EliahKagan commented 2 months ago

What's the point using mac runners for Docker tasks if you can use Linux runners?

Currently there are no GitHub-hosted ARM-based Linux runners.

hovancik commented 3 weeks ago

@fwcd @cgrindel were you able to fix errors with xcrun --show-sdk-version?

cgrindel commented 2 weeks ago

Unfortunately, no. I still see the following:

++ xcrun --show-sdk-version
2024-04-28 15:28:55.452 xcodebuild[11203:62473] Writing error result bundle to /var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/ResultBundle_2024-28-04_15-28-0055.xcresult
xcodebuild: error: SDK "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk" cannot be located.
xcrun: error: unable to lookup item 'SDKVersion' in SDK '/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk'
hovancik commented 1 week ago

I've created issue #9811

joffrey-bion commented 4 days ago

What's the point using mac runners for Docker tasks if you can use Linux runners?

Sometimes Docker is just here to provide a side service while running native tests that require a specific system. I want to test my web socket client implementation on macOS, not on linux, but I still want to run my docker-based test server during the build.