Ubuntu-latest workflows will use Ubuntu-24.04 image

[ijunaidm]

Rollout will begin on December 5th and will complete on January 17th, 2025.

Breaking changes Ubuntu 24.04 is ready to be the default version for the "ubuntu-latest" label in GitHub Actions and Azure DevOps.

Target date This change will be rolled out over a period of several weeks beginning December 5th and will complete on January 17th, 2025.

The motivation for the changes GitHub Actions and Azure DevOps have supported Ubuntu 24.04 in preview mode since May 2024, and starting from July 2024 Ubuntu 24.04 is generally available for all customers. We have monitored customer feedback to improve the Ubuntu 24.04 image stability and now we are ready to set it as the latest. There are a set of packages listed below that we have removed from the Ubuntu 24 image. Please review the list carefully to see if you will be impacted by these changes. We have made cuts to the list of packages so that we can maintain our SLA for free disk space. The images have grown so large we are in danger of violating our SLA if we keep the package list as-is.

The factors we took into consideration when removing packages are as follows:

• How long does it take to install the tool at runtime?
• How much space does it take up on the image?
• How many users are there of the tool?

We understand that our reasoning may not make sense to some of you out there, but please bear in mind that we tried to keep disruptions as minimal as possible, and tried to keep the best interests of the community at large in mind. There is a very large and diverse community using our images, and as much as we would like to, we cannot pre-install every tool on these images.

Platforms affected

• [x] Azure DevOps
• [x] GitHub Actions

Mitigation ways Steps or options for impact mitigation If you see any issues with your workflows during transition period:

• Switch back to Ubuntu 22 by changing workflow YAML to use runs-on: ubuntu-22.04 We support two latest LTS Ubuntu versions, so Ubuntu 22 will still be maintained for the next 2 years.
• File an issue in this repository

Software Differences The Ubuntu 22.04 image has a different set of software than Ubuntu 24.04. The most significant changes are listed in the table below:	Tool name	Ubuntu 22.04	Ubuntu 24.04	Notes
Clang	13. 14. (default) 15.*	16. 17. 18.* (default)	The most recent versions are installed
GCC / GNU C++ / GNU Fortran	9. 10. 11. 12. 13.*	12. 13. 14.*	The most recent versions are installed
PHP	8.1.*	8.3.*	The most recent version are installed
Java	8. 11. (default) 17. 21.	8. 11. 17. (default) 21.	Default Java switched to 17.* for Ubuntu 24.04 image.
Python	3.7. (cached) 3.8. (cached) 3.9. (cached) 3.10. (default) 3.11. (cached) 3.12. (cached)	3.9. (cached) 3.10. (cached) 3.11. (cached) 3.12. (default)	Pre-cached versions currently unavailable. Default version switched to the latest one. On GitHub Actions, actions/setup-python can install any version on-flight so this change doesn't impact users
Go	1.20. (cached) 1.21. (default) 1.22.* (cached)	1.21. (cached) 1.22. (cached) 1.23.* (default)	If your use-case requires using any of these versions, consider using tasks to install Go on-flight: actions/setup-go (GitHub Actions) Go Tool Installer (Azure DevOps)
PyPy	3.7. (cached) 3.8. (cached) 3.9. (cached) 3.10. (cached)	3.9. (cached) 3.10. (cached)	Deprecated all the versions besides the most recent ones
Ruby	3.0. (default) 3.1. (cached)	3.2.* (default)	We need to update 3.3.5
Node.js	16. (cached) 18. (default) 20.* (cached)	16. (cached) 18. (cached) 20.* (default)	If your use-case requires using any of these versions, consider using tasks to install Node.js on-flight: actions/setup-node (GitHub Actions) Node.js Tool Installer (Azure DevOps) Node 16 will be replaced with Node 22
Heroku	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Leiningen	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Mono / MSBuild / NuGet	latest available	-	Software is not available for Ubuntu 24 at the moment.
Terraform	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
R	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
SVN	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Alibaba Cloud CLI	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Netlify CLI	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
OpenShift CLI	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
ORAS CLI	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Vercel CLI	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Bindgen / Cbindgen	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Cargo audit/clippy/outdated	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
.NET Core SDK	6. 7. 8.*	8.*	Please consider using tasks to install any version on-flight: actions/setup-dotnet (GitHub Actions) Use .NET Core (Azure DevOps)
PostgreSQL	14.*	16.*	More recent version are installed
MS SQL Server Client Tools	sqlcmd / SqlPackage	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
MarkdownPS Module	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Android Command Line Tools	9.0	12.0	The most recent version are installed
Android SDK Build-tools	34.0.0 33.0.0 33.0.1 33.0.2 33.0.3 32.0.0 31.0.0	34.0.0	The most recent version are installed
Android NDK	25. (default) 26.	27. (default) 26.	The most recent version are installed
Cached Docker images	alpine:3.16 alpine:3.17 alpine:3.18 debian:10 debian:11 moby/buildkit:latest node:16 node:16-alpine node:18 node:18-alpine node:20 node:20-alpine ubuntu:20.04 ubuntu:22.04	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.

[ThibaultLesuisse]

Nuget and sqlpackage not being available is a major showstopper for us.

The more I look at this list the more I realize that 24.04 is a step back for a lot of people. People are going to complain when this becomes the ubuntu-latest. Be aware of this and maybe make sure this communication is shown on github/azure devops

But thanks for the heads up!

[MikeMcC399]

@ijunaidm

ubuntu-24.04 is still labeled beta in the overview on https://github.com/actions/runner-images/blob/main/README.md#available-images

Will you now be removing the beta tag?

[ijunaidm]

Yes @MikeMcC399 . Its updated and removed.

[zivkan]

@ThibaultLesuisse are you aware that NuGet refers specifically to NuGet.exe (which requires mono on Mac & Linux), as does Azure Pipeline's NuGetCommand, and GitHub Actions' nuget/setup-nuget action?

If you're building your projects with dotnet build, you're much better off restoring with dotnet restore, and not using NuGet.exe or mono at all.

[ThibaultLesuisse]

But we need NuGet.exe to sync our custom Artifacts store. And yes I know that commonly you need to run mono nuget.exe to run it on Mac/Linux. But it doesn't change the fact that a lot of tools are now missing for various reasons. This should be communicated.

[PathogenDavid]

We're also affected by the removal of Mono.

The official Mono repo only lists 20.04, but the package seem to work fine in newer versions–which is how the 22.04 image got Mono:

https://github.com/actions/runner-images/blob/ae99c16b0cf27c0cca7ef0b08e7bcf13c33f6cfa/images/ubuntu/scripts/build/install-mono.sh#L15-L18

Is there a reason this can't be done for 24.04 as well? I tested it briefly and it's working for our relatively simple needs.

Mono is fairly chunky so we'd rather not install it every single workflow run.

[sayhiben]

Hi there - my team is scrambling this morning due to this change. I have detailed the issue on the community forum here, but the tl;dr is that this version bump prevents Python packages from being installed for system Python. Unfortunately, it seems the evaluation of this issue was incorrect:

On GitHub Actions, actions/setup-python can install any version on-flight so this change doesn't impact users

In fact, my team has a large number of workflows created by many different engineers, and around a dozen of our workflows that didn't previously leverage actions/setup-python now have to go through our internal change management process in order to restore stability to our CI/CD

In the future, please announce breaking changes as a deprecation notice on the GitHub Blog. It's the one place we rely on to proactively catch and prevent stability issues from occurring in our pipelines

[obluff]

This morning, several of our CI pipelines broke due to these changes. It would be great to have compatibility with the following dependencies, allowing us to upgrade the ubuntu version in the future without complicating our pipelines :bow: .

• `gcloud
• `sbt
• libncurses.so.5

[aylapear]

We have had issues today due to the change. We are getting this error message when using ubuntu-latest

 unable to load shared object '/home/runner/work/_temp/Library/ragg/libs/ragg.so':
  libtiff.so.5: cannot open shared object file: No such file or directory

When used the previous version ubuntu-22.04 everything runs without issue. I am unsure of what I need to update in our Action scripts to fix this issue.

[oc-liz-conlan]

I think I'm experiencing the same problem as @sayhiben - without either adding an extra step to a Ruby workflow to install an older Python version or deliberately requesting ubuntu-22.04 to avoid the problem completely, any GitHub Package installer that uses pip to install system-level packages inside the container is going to fail with the following output:

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.12/README.venv for more information.

[At252]

-Readme.md

[mcmillan03]

I am trying to use arm-runner-action with the base specified at ubuntu-latest. Sometime in the last two weeks my workflow for arm platform started failing because qemu was not getting configured/built?/whatever correctly for the new ubuntu-latest. The work around is to specify ubuntu-22.04. The error using ubuntu-latest with arm-runner-action@v2.6.5 is:

...
Setting up g++ (4:10.2.1-1) ...
update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode
Setting up dpkg-dev (1.20.13) ...
Setting up build-essential (12.9) ...
Processing triggers for libc-bin (2.31-13+rpt2+rpi1+deb11u10) ...
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)
dpkg: error processing package libc-bin (--configure):
 installed libc-bin package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
 libc-bin
E: Sub-process /usr/bin/dpkg returned an error code (1)
Error: Process completed with exit code 100.

[lukeschlather]

Removing gcloud is going to break a bunch of our pipelines, this is a pretty major breaking set of changes to remove all these packages.

[eskultety]

Would you please kindly update the README and put the ubuntu-latest YAML label back to 22.04 until the migration is fully over and this PR is closed? We found this the hard way in our workflow when we assumed a version of one of the tools only to realize the underlying host was something else than advertised.

[Loki-Afro]

to add something positive: looking forward to an updated curl version by default

[mdirkse]

Our pipelines broke this morning because Terraform is no longer included. While I have no issue with it being removed (these are GitHub's images and I'm consciously risking my pipelines breaking by relying on ubuntu-latest instead of pinning) it would be nice to have a little more explanation as to why TF was removed (and tools like bicep and packer left in). It's now just in a list of other removed tools with the remark "Maintenance reasons". Can anyone elaborate?

[telmo-rodrigues-yld]

Is there an actual reason as to why nuget cli was removed?

[pkuczynski]

We were suddenly hit by removed terraform too... Any particular reason for it? Is it gonna be a permanent change or will you work on bringing it back?

[mikehardy]

brainstorm idea: would be nice to have <image type>-beta labels available for use in periodic workflows

I usually leave the images specified as <image type>-latest and accept that my workflows will maybe break without notice, because the changes are usually easy to handle and it keeps my repos current

But since the upcoming releases are just tagged by specific version numbers until the latest tag slides to them, there's no way to have a stable workflow that pre-tests things and finds the breaks before they hit our important pipelines

If there was a stable name I could do matrix expansion on for scheduled checks, it might help folks doing their best to stay current but without unexpected issues

Cheers

[t1m0thyj]

This morning several of our CI pipelines broke as well because they access the Linux keyring. It would nice to have the libsecret-1-dev dependency restored in Ubuntu 24 since it was part of the image for previous versions.

[mikehardy]

Not sure if this is now the place to ask or not but since others are, we were using liblzma I suppose with the removal of a huge huge pile of packages, all their transitives are gone as well which is why the libraries are gone. Not too hard to install them at least.

[rschwarz]

~Similarly, I think that liblapack3 was lost during the upgrade from 2204 to 2404.~

Sorry, I take that back. The library was simply updated, but we used a binary that was looking for an older version of the shared library yielding the error. We worked around the issue simply by fixing the Ubuntu version for now.

[kojiromike]

See #10783

[molson504x]

Referencing this issue here - #10781 is caused by this. Documentation should be updated to reflect this breaking change, or the interpreter of python that comes with the runner image should be installed independently from the interpreter that ships with ubuntu so that it is no longer externally managed.

This change WILL break a lot of actions and existing workflows which target the ubuntu-latest label.

[itsdouges]

Hello! This update has broken my ubuntu end-to-end tests with Playwright + Electron. Need to investigate why. For now I'll pin to runs-on: ubuntu-22.04.

[Saibamen]

If you're building your projects with dotnet build, you're much better off restoring with dotnet restore, and not using NuGet.exe or mono at all.

And what about nuget pack? It crashes even if I'm using dotnet restore and dotnet build

[ljharb]

See https://github.com/actions/runner-images/issues/10788 - the new image fails to include nvm and sbt. (setup-node is not a sufficiently capable replacement for nvm, ftr)

Please reverse some of these very ill-advised breaking changes, and please consider using notification to alert any repo owners that would be affected by these kinds of breaking updates.

Especially since everyone's been suggested to use "latest" for years, it's just not acceptable to ever ship breaking changes that aren't absolutely necessary.

[Ryangr0]

This broke all of our playwright tests. I guess you have to crack a few eggs to make an omelet, but I'm missing some "official" word on what is going on. From what I understand, all ubuntu-latest now run on 24.04, and it is fundamentally broken. Is this being worked on? Pinning everything we have running on non-private runners to 22.04 for now...

[jajik]

From what I understand, all ubuntu-latest now run on 24.04, and it is fundamentally broken.

Unfortunately, that's not true. Some of the ubuntu-latest run on 24.04, others are still 22.04 even though the documentation says something else. I wouldn't say 24.04 is fundamentally broken, it just differs from 22.04 and the communication is far from ideal.

[Saibamen]

From what I understand, all ubuntu-latest now run on 24.04, and it is fundamentally broken.

Unfortunately, that's not true. Some of the ubuntu-latest run on 24.04, others are still 22.04 even though the documentation says something else. I wouldn't say 24.04 is fundamentally broken, it just differs from 22.04 and the communication is far from ideal.

Yes. On master repo, ubuntu-latest runs on Ubuntu LTS 24.04.1, but on my fork it runs in 22.04.5 (see: https://github.com/NuGet/setup-nuget/issues/168).

[kdaigle]

The team is actively rolling this back, and we expect it to be done in about an hour. Sorry for the pain this caused in breaking you all. We're going to ensure that we don't break you in unexpected ways, announcements aside.

I'll let the teams share any more pertinent details but wanted to come share an update quickly. ⚡ 🙇

[lkfortuna]

Hey everyone - thank you for your input. Based on community feedback we have decided to rollback the ubuntu-latest migration. All runs on ubuntu-latest will go back to Ubuntu-22 while we work out the issues with the image. We apologize for any service disruption you all may have had, and appreciate everyone’s feedback and patience. The rollback will be completed in the next hour.

To give some context for the changes we made - our images have become very large over the past 2 years and we were trying to free up some space to give us some breathing room going forward. There is currently no extra free space left for installing any more tooling, while still maintaining our free space SLA of 14gb. The size of the image also impacts performance of the VMs as well. Since a new OS is always a cutover, we took the opportunity to free up some space. Clearly this was not the right approach, and we apologize for the pain we have caused.

We will re-evaluate our communication and rollout strategy before beginning this migration again.

[geekosaur]

One request re communication: currently it seems that to know about upcoming changes like this, you need to subscribe to this repo, but there isn't a way to filter that to only announcements so I'm getting a lot of unrelated messages in my inbox. A better communication channel for image updates would be appreciated.

[voxpelli]

@geekosaur Its announced in the GitHub Changelog: https://github.blog/changelog/2024-09-25-actions-new-images-and-ubuntu-latest-changes/

It has eg RSS and also has RSS for specific tags, so you can eg. subscribe to all actions tagged changelog items through: https://github.blog/changelog/label/actions/feed/

That will get you all GitHub Actions related announcements and it should have given you this info, but this time it actually didn’t contained enough information, see: https://github.com/actions/runner-images/issues/10788#issuecomment-2416557822

[ljharb]

Huge thanks to github for listening to feedback! I'm more than happy to help brainstorm ideas to reduce the image size in a non-breaking way.

[molson504x]

Hey everyone - thank you for your input. Based on community feedback we have decided to rollback the ubuntu-latest migration. All runs on ubuntu-latest will go back to Ubuntu-22 while we work out the issues with the image. We apologize for any service disruption you all may have had, and appreciate everyone’s feedback and patience. The rollback will be completed in the next hour. ...

Thank you for the transparency on this issue and a swift resolution @lkfortuna and @kdaigle :)

[WardBrian]

Ironically, my CI worked fine under 24.04, but when the revert occurred I started seeing some weird errors about GLIBCXX versions.

It turns out, the caches were populated under 24.04. If anyone else runs into this, I was able to resolve by clearing the cache for that job. This is perhaps a danger of caching based on the runs-on key, but I don't know if there is an easy way to get the os version from the environment

[AlfishSoftware]

It's probably better to change all documentation to say using *-latest is strongly discouraged, at least if you're using any tool outside of a very basic set that is guaranteed to always be available (e.g. apt and setup actions). Even then, it's probably better to encourage everyone to just pin it.

Or have something like e.g. ubuntu-22.04-or-later (or just use this "or later" logic on regular ubuntu-22.04) which uses the specified version if available, otherwise automatically use whichever earliest next version is available (not sure if it's already like this) to let everyone pin it with the least trouble.

it seems that to know about upcoming changes like this, you need to subscribe to this repo, but there isn't a way to filter that to only announcements It has eg RSS and also has RSS for specific tags

RSS is not great for communication... the ideal would be for everyone who has authored any action with ubuntu-* to receive email notifications once a new Ubuntu version is available and production-ready, so they see if it's worth manually migrating it. You could add a setting to the profile to disable that if desired. This notification can be enabled for everyone by default, because it's not like it's gonna happen e.g. every month. It's very infrequent, right?

[voxpelli]

RSS is not great for communication... the ideal would be for everyone who has authored any action with ubuntu-* to receive email notifications once a new Ubuntu version is available and production-ready, so they see if it's worth manually migrating it

The availability of this release was announced months ago, people should ideally have tested it then and the list of changes in this issue should have been attached or linked to from that announcement.

Automated update notifications for runner versions can be configured in Renovate (and possibly Dependabot, I don’t know)

A new LTS of Ubuntu is released every other year, so that specifically is not a high frequency, and is supported for 5 years

A new ubuntu runner image should probably be in beta like now for 3-6 month and be out of beta for at least another 3-6 months before becoming the default image. That leaves enough room for the community to adapt.

[Ryangr0]

You don't know what you don't know. It's easy to get blindsided. It's not necessarily anyone's fault. I've found it's easier to avoid these types of things by just never relying on latest.

[mikehardy]

@voxpelli it would be easier to test and have a canary workflow if there were a sliding -beta tag, I use -latest but it is difficult as maintainer in a bunch of repos to go around manually changing them all to a numbered release tag that will then be obsolete after

[ljharb]

Additionally, there's an existing deprecation message vehicle for actions that could be used - the warnings that pop up when you're indirectly using an about-to-be-removed node runtime, for example.

[voxpelli]

would be easier to test and have a canary workflow if there were a sliding -beta tag

That would be great! Create an issue if there is none! Maybe -next would be even better? Or both? ubuntu-next? And -latest would have been better named -default, -current, -stable or such

the warnings that pop up when you're indirectly using an about-to-be-removed node runtime, for example

Those changes are breaking changes within an existing runner image I believe, not about moving to a new image

[geekosaur]

If ubuntu-latest were actually "stable" then we wouldn't be here right now. It's correctly named latest, and arguably 22.04 is stable.

[jajik]

If ubuntu-latest were actually "stable" then we wouldn't be here right now. It's correctly named latest, and arguably 22.04 is stable.

Providing different packages or not providing others is not about stability.

[pkuczynski]

If smaller image would mean that my workflows start much faster, I would be fine installing the deps myself, especially if installation is shorter than time saved on starting the job. Unfortunately I have not observed this…

What I liked in gitlab CI was that it allowed you to use any docker image. Which would mean anybody could prepare an image shaved to his needs. But I guess than you would run the trouble of caching those images on the runners…?

[torokati44]

For one of our workflows, it was the rollback to 22.04 that broke things... :innocent:

[ultimaweapon]

I though it was GitHub bugs that rolling -latest back to 22.04, which break some of my workflows that I already upgraded. I should always using specific version instead of -latest from now on.

[jr0me]

If the action/workflow is set to a value that doesn't exist, will it fail? Or just wait? I changed ubuntu-latest to ubuntu-24 and... it's queued forever.

[ljharb]

@jr0me it should be ubuntu-24.04 i believe.