actions / runner-images

GitHub Actions runner images
MIT License
9.17k stars 2.84k forks source link

macOS 11.0 pools will be transited to private preview. #2486

Closed AlenaSviridenko closed 2 years ago

AlenaSviridenko commented 3 years ago

Hi GitHub Actions community!

A couple of months ago we added a new macOS 11.0 image in a “preview” mode and eventually we expect it to become a production image.

Typically, we increase pool capacity along with increasing image usage, but demand for this pool is outpacing our capacity plan. Given the constantly growing demand and in order to prevent degraded experience, we have to temporarily transit this pool to a private preview until we come up with a final plan of making macOS 11.0 ready for production use.

What does “private preview” mean? This limitation will live on organization level (personal accounts are considered as organization too), so if you are using macos-11 somewhere in your organization — you will still be able to use it. But this pool will be inaccessible for new organizations that never used it before.

We are sorry for inconveniences, and will try to open this pool again as soon as possible.

UPDATE

The macOS 11 virtual environment is currently in preview and is automatically available to the existing Enterprise plan customers who used macOS hosted runners at least once between May, 1 - June, 1. New Enterprise plan customers, or customers on other plans, should fill the form to request access to macOS 11 virtual environment. Please view our Big Sur guide for more details.

tomhamming commented 3 years ago

The docs still mention macos-11.0:

https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idruns-on

AlenaSviridenko commented 3 years ago

Hi @tomhamming, thanks for noticing it, we will update docs soon as well.

RobbieTheWagner commented 3 years ago

Would this have anything to do with my macos builds always hanging now?

AlenaSviridenko commented 3 years ago

Hi @rwwagner90, closing pool was the first step in resolving constantly increasing queues, and now we are looking into how to improve current queues given the capacity we have. Hopefully, queues will become more healthy soon. Thank you for your patience.

RobbieTheWagner commented 3 years ago

@AlenaSviridenko thanks, hope things improve soon! I wasted a lot of hours yesterday trying to debug this, but I think it was just the Mac images not working.

paulz commented 3 years ago

macOS 11 image works great for us. We are using GitHub actions as our CI to build and release macOS 11 app.

When the preview is planned to be a production?

AlenaSviridenko commented 3 years ago

Hi! Currently, we don't have any certain ETA, but we will announce officially and publicly when macOS 11.0 will be ready for production.

christian-kreuzberger-dtx commented 3 years ago

Hi, big thanks for the ongoing support for macos 11.0, we're using it in keptn project for building our CLI.

coming from travis-ci, I always liked this status page https://www.traviscistatus.com/, most importantly the "backlog" (= queue) of macos builds: image

Is there such a thing for virtual environments on GH Actions? This would really help users to troubleshoot problems (e.g., oh my pipeline is stuck because MacOS builds are queued)

paulz commented 3 years ago

Thank you for supporting latest macOS release. As a customer we would like GitHub to make public releases of runners when Apple makes macOS release public. For example macOS 11 was released November 2020. Meantime we are happy to use beta or preview versions.

KeiroMidori commented 3 years ago

How does one know if they are being part of new or existing organizations using macos-11.0 ? Will there be verbose errors saying that they should use macos-latest instead?

radarhere commented 3 years ago

How does one know if they are being part of new or existing organizations using macos-11.0 ? Will there be verbose errors saying that they should use macos-latest instead?

Over at https://github.com/python-pillow/pillow-wheels/runs/1813985728?check_suite_focus=true, I'm see 'X This check failed' - contrasted with the successful run at https://github.com/radarhere/pillow-wheels/runs/1813987594?check_suite_focus=true, where my fork is in the pool and runs without a problem.

AlenaSviridenko commented 3 years ago

How does one know if they are being part of new or existing organizations using macos-11.0 ? Will there be verbose errors saying that they should use macos-latest instead?

Hi! if organization used macos-11.0 last 30 days — pool should be available. If it is not — No runner matching the specified labels was found: macos-11.0. error will be displayed.

KeiroMidori commented 3 years ago

How does one know if they are being part of new or existing organizations using macos-11.0 ? Will there be verbose errors saying that they should use macos-latest instead?

Hi! if organization used macos-11.0 last 30 days — pool should be available. If it is not — No runner matching the specified labels was found: macos-11.0. error will be displayed.

Roger that. Apparently my organization is able to use macos-11.0 but it's really unstable. Queue times are enormous and workflows often fail. I went back to using macos-latest for now but I'd love to see these private previews actually working for those who can use them.

AlenaSviridenko commented 3 years ago

Hi @christian-kreuzberger-dtx, unfortunately GH Actions doesn't provide such status page, but the idea is interesting, we'll take a look into it, thanks!

erikng commented 3 years ago

@AlenaSviridenko my organization had used the macos-11.0 image (11 times) with a latest use of 2 months ago. We maintain a custom python that is used for tens of thousands of enterprise macOS devices. We hadn't had to build python until today and now it will be impossible to maintain the project on GitHub.

I need access to the image because it's the only way to reliably build universal macOS python wheels that support both Intel and Apple Silicon macs.

Is there anyway to get this re-allowed? The project is located here: https://github.com/macadmins/python

Also, per your initial comment in the issue we should still have access:

What does “private preview” mean?
This limitation will live on organization level (personal accounts are considered as organization too), so if you are using macos-11.0 somewhere in your organization — you will still be able to use it. But this pool will be inaccessible for new organizations that never used it before.
paulz commented 3 years ago

we are waiting for a long time for runner to pick up jobs on macos-11.0 to run our CI tests.

if it helps for anyone we found that macos-latest runner can at least compile and build MacOS 11.0 application. So we are using more macos-latest runner to build and publish releases for macOS 11 even it is unable to run tests or even load the app.

csett86 commented 3 years ago

@erikng if it helps, this is how I built universal binaries with the current macos-latest (10.15) runner: https://github.com/csett86/jitsi-meet-electron/blob/main/.github/workflows/nodejs.yml#L58

wez commented 3 years ago

FWIW, I suspect that a lot of CI use cases just want to build with the newer SDK (mine fit in this category), so if the docs made it clearer how to do that, then I suspect that there would be less demand for 11.0 itself.

dennisameling commented 3 years ago

@erikng if it helps, this is how I built universal binaries with the current macos-latest (10.15) runner: https://github.com/csett86/jitsi-meet-electron/blob/main/.github/workflows/nodejs.yml#L58

That works, thanks @csett86! 🚀

erikng commented 3 years ago

The problem is two fold for us. One is compiling the wheels and then the secondary is creating the package. From how I understand it, only productbuild on Big Sur can create a package with a distribution file that supports both Intel and Apple silicon Macs. If you build a package on Catalina, it will be universal (and can be notarized) but it will trigger a user to need to install Rosetta2 on AS Macs.

Those tools are built into the OS and not part of the Xcode command line tools package.

tonyarnold commented 3 years ago

GitHubbers, could you please provide an update on this issue? It's been 3 months since Big Sur's public release - I understand that there may be extenuating circumstances here, but if this is the speed with which the Actions team is going to be deploying major macOS updates, then Hosted Runners aren't going to be a viable option for those of us trying to use your service to build macOS apps.

As I'm sure you are also aware, Xcode 12.5 is now in beta, and it requires macOS 11. Things are about to become untenable for many iOS developers using GitHub Actions who want to test on the latest SDKs.

AlenaSviridenko commented 3 years ago

Hello,

we apologize for a long silence, we understand that macOS 11.0 is a highly demanded OS now because of the latest Xcode, unique Swift UI features and so on, this is our 1st priority for now, and although it appeared to be a pretty tricky from the all point of view, we are almost on the final line with the support and capacity plan. We will come up with this plan by the end of February.

Thank you for your patience.

jpsim commented 3 years ago

We will come up with this plan by the end of February.

Thanks for your last update @AlenaSviridenko. Are there any details you can share about this plan?

AlenaSviridenko commented 3 years ago

Hi there, sad to say, but not too many updates. Unfortunately, we still do not have the ability to take new users onto the preview pool. We expect the earliest we can is in April, when we're first able to start adding new capacity.

QuiiBz commented 3 years ago

We will wait - but is it normal to still have macos-11.0 listed in the Supported runners? I think it should advertise that no new users can be added to the preview pool.

AlenaSviridenko commented 3 years ago

There is a note below for that: but we're not accepting any further users or organizations at this time.. We still need to refer to Readme files, that is why we have macOS 11 on that page.

erikng commented 3 years ago

@AlenaSviridenko is there any chance you could help my org? We host packages for enterprise customers around the world. As I stated before we had access to the runners and per notes from GitHub all customers that had run them prior would not be impacted but when the change was deployed, that was not the case.

AlenaSviridenko commented 3 years ago

Hi @erikng, we included your org back to macOS-11.0 pool, sorry for inconveniences and late reply. We have some internal limitation of pool usage history, that is why your org was not included previously.

erikng commented 3 years ago

Thank you!

EvgeneOskin commented 3 years ago

Hi @AlenaSviridenko, we've run GitHub Action with macOS 11 runner before it moved to private preview. We used it to distribute the Termius app for the arm64 architecture. Can we get access back to the macOS-11.0 pool? Could you help with this?

kalcutter commented 3 years ago

Please give us back access to macOS 11. Our organisation was using it successfully last year.

sarahlovemike commented 3 years ago

I need help don't know what I'm doing

kode54 commented 3 years ago

Guess I'll just continue doing builds on my personal machine, and set my CI to only test Intel builds.

EwoutH commented 3 years ago

@AlenaSviridenko Could you give a status update on the macOS 11.0 image?

freak4pc commented 3 years ago

Sorry for jumping late to the party but this is quite silly. At the bare minimum you should remove this information from virtual-environments to avoid confusion and time waste :)

AlexanderAmelkin commented 3 years ago

I second freak4pc's suggestion. I must note also that if I add macos-11.0 and want to use a matrix of [ maxos-10.15, macos-11.0 ], then the 11.0 build never finishes without saying that it is not available, and the 10.15 build silently gets 'cancelled' without any explanation at all. That doesn't look like correct behavior. I think you must announce removal of 11.0 environment for new users, and remove 11.0 from the documentation until it the capacity is available. The current behavior is highly confusing.

MarcoEidinger commented 3 years ago

@AlenaSviridenko Would you be so kind to give a status update on the macOS 11.0 image? Today's Xcode 12.5 RC release (on macOS 11 only) makes the availability even more critical as the official release is expected for next week.

AlenaSviridenko commented 3 years ago

Hi 👋, we're working on a plan to get Big Sur added to the hosted pool sometime in June/July, we are still working on hardware infrastructure to be ready. We are aware about stable Xcode 12.5 coming soon and doing everything possible to make macOS 11 ready as soon as possible.

christian-kreuzberger-dtx commented 3 years ago

Seems like MacOS 11 builds are currently not working even in private preview. They are failing with

An error occurred while provisioning resources (Error Type: Failure).
sisoje commented 3 years ago

Hi 👋, we're working on a plan to get Big Sur added to the hosted pool sometime in June/July, we are still working on hardware infrastructure to be ready. We are aware about stable Xcode 12.5 coming soon and doing everything possible to make macOS 11 ready as soon as possible.

that is way too late

freak4pc commented 3 years ago

Three months(!!) for a new version of Xcode? That would be a month after WWDC. If you're trying to be an acceptable commercial CI solution, this kind of SLA will never work for developers.

Bitrise already has Xcode 12.5 RC1.

tcldr commented 3 years ago

we're working on a plan to get Big Sur added to the hosted pool sometime in June/July

This is very frustrating for those of us who have invested significant time into GitHub Actions.

Should we expect a similar lead time for macOS 12 when it is inevitably announced in June/July?

danpalmer commented 3 years ago

There was a big licencing change (#2604) between macOS 10.15 and 11, specifically around how machines can be rented out. I'd assume that GitHub/Microsoft are probably building out infrastructure based on the new licencing (I'd expect to see Azure macOS VMs), and that will back Azure Pipelines and GitHub Actions once it's live. If this is the case this delay is likely a one-off issue rather than a pattern for the future.

I suspect this is not something we need to worry about, although it would certainly be nice to see it happen as soon as possible.

Anyone at GitHub/Microsoft please do correct me, if you're allowed to!

Edit: looks like I was over-thinking this and it is just time to onboard new OS versions.

AlenaSviridenko commented 3 years ago

Should we expect a similar lead time for macOS 12 when it is inevitably announced in June/July?

Hopefully, not. Initially, GitHub Actions started with a single macOS version, so adding a new one brought some challenges. But once we polish the onboarding plan, we could stick to it for future versions, so onboarding cadence should be way better.

lrun-cmck commented 3 years ago

Without macOS 11.0, we won't be able to use Xcode 12.5 which requires Big Sur 11 or later (https://developer.apple.com/documentation/xcode-release-notes/xcode-12_5-release-notes)

scannillo commented 3 years ago

Without macOS 11.0, we won't be able to use Xcode 12.5 which requires Big Sur 11 or later (https://developer.apple.com/documentation/xcode-release-notes/xcode-12_5-release-notes)

Also worth adding we need Big Sur in order to test iOS 14.5 and up.

ffried commented 3 years ago

Without macOS 11.0, we won't be able to use Xcode 12.5 which requires Big Sur 11 or later (https://developer.apple.com/documentation/xcode-release-notes/xcode-12_5-release-notes)

Also worth adding we need Big Sur in order to test iOS 14.5 and up.

Yep. Also, it makes it a lot harder to use Swift 5.4 on macOS. I've hit this today while updating our OSS components - which are now all untested on the newest Xcode / Swift 5.4.

rmaloney3 commented 3 years ago

I agree with all of the comments said thus far on the topic. macOS 11 needs to be opened up much sooner than June/July. We need the ability to use Xcode 12.5 on CI, as we have imminent plans to start using Swift 5.4 features.

I should also add that Apple has already released a beta version of iOS 14.6 which also relies on Xcode 12.5. We also all know that at WWDC on June 7, Apple will be releasing betas for Xcode 13, macOS 12, iOS 15, etc. These need to be available on CI before July as well.

I have to say I expected more from GitHub and Microsoft.

nzjrs commented 3 years ago

What the hell? The docs still mention this as available and yet I have to dig around here to find out that the semi-silent failure of my matrix build on macos-11 was actually by design.

michaelpeternell commented 3 years ago

Sorry, maybe I am saying something obvious now..

Do you know that you don't need Apple Silicon to run macOS 11?

I would of course appreciate a virtual environment with macOS 11.3 running on Apple Silicon. But a virtual environment with macOS 11.3, running on an Intel Mac, with Xcode 12.5, would be fine for now, at least for me. And it would certainly be better than no macOS 11 support at all.