Ubuntu 24.04 is now available

mikhailkoliada commented 1 month ago

Breaking changes

Ubuntu 24.04 is now available for all GitHub Actions and Azure DevOps users. You can use the ubuntu 24.04 image label in your YAML to select this image.

GitHub Actions

jobs:
  jobName:
    runs-on: ubuntu-24.04

Azure DevOps

pool:
   name: 'Azure Pipelines'
   vmImage: 'ubuntu-24.04'

The image is marked as "beta" for now. It means some software can be unstable on the new platform. Also, there could be queueing issues as the capacity will be balanced only throughout the next weeks. Please report any problems with the new image to this repository.

Platforms affected

[X] Azure DevOps
[X] GitHub Actions

Runner images affected

[ ] Ubuntu 20.04
[ ] Ubuntu 22.04
[X] Ubuntu 24.04
[ ] macOS 11
[ ] macOS 12
[ ] macOS 13
[ ] macOS 13 Arm64
[ ] macOS 14
[ ] macOS 14 Arm64
[ ] Windows Server 2019
[ ] Windows Server 2022

Software difference between Ubuntu 22 and Ubuntu 24

Tool name	Ubuntu 22.04	Ubuntu 24.04	Notes
Clang	13. 14. (default) 15.*	16. 17. 18.* (default)	The most recent versions are installed
GCC / GNU C++ / GNU Fortran	9. 10. 11. 12. 13.*	12. 13. 14.*	The most recent versions are installed
Kotlin	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
PHP	8.1.*	8.3.*	The most recent version are installed
Swift	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Java	8. 11. (default) 17. 21.	8. 11. 17. (default) 21.	Default Java switched to 17.* for Ubuntu 24.04 image.
Python	3.7. (cached) 3.8. (cached) 3.9. (cached) 3.10. (default) 3.11. (cached) 3.12. (cached)	3.9. (cached) 3.10. (default) 3.11. (cached) 3.12. (default)	Pre-cached versions currently unavailable. Default version switched to the latest one. On GitHub Actions, actions/setup-python can install any version on-flight so this change doesn't impact users
Go	1.20. (cached) 1.21. (default) 1.22.* (cached)	1.20. (cached) 1.21. (cached) 1.22.* (default)	If your use-case requires using any of these versions, consider using tasks to install Go on-flight: actions/setup-go (GitHub Actions) Go Tool Installer (Azure DevOps)
PyPy	3.7. (cached) 3.8. (cached) 3.9. (cached) 3.10. (cached)	3.9. (cached) 3.10. (cached)	Deprecated all the versions besides the most recent ones
Ruby	3.0. (default) 3.1. (cached)	3.2.* (default)	Pre-cached versions currently unavailable. Default version switched to the latest one.
Node.js	16. (cached) 18. (default) 20.* (cached)	16. (cached) 18. (cached) 20.* (default)	If your use-case requires using any of these versions, consider using tasks to install Node.js on-flight: actions/setup-node (GitHub Actions) Node.js Tool Installer (Azure DevOps)
Heroku	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Leiningen	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Mono / MSBuild / NuGet	latest available	-	Software is not available for Ubuntu 24 at the moment.
Packer / Terraform	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
R	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
SVN	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Alibaba Cloud CLI	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Google Cloud CLI	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Netlify CLI	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
OpenShift CLI	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
ORAS CLI	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Vercel CLI	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Bindgen / Cbindgen	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Cargo audit/clippy/outdated	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Microsoft Edge	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Microsoft Edge WebDriver	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Mozilla Firefox	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Mozilla Geckodriver	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
.NET Core SDK	6. 7. 8.*	8.*	Please consider using tasks to install any version on-flight: actions/setup-dotnet (GitHub Actions) Use .NET Core (Azure DevOps)
PostgreSQL	14.*	16.*	More recent version are installed
MS SQL Server Client Tools	sqlcmd / SqlPackage	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
MarkdownPS Module	latest available	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.
Android Command Line Tools	9.0	12.0	The most recent version are installed
Android SDK Build-tools	34.0.0 33.0.0 33.0.1 33.0.2 33.0.3 32.0.0 31.0.0	34.0.0	The most recent version are installed
Android NDK	25. (default) 26.	26.* (default)	The most recent version are installed
Cached Docker images	alpine:3.16 alpine:3.17 alpine:3.18 debian:10 debian:11 moby/buildkit:latest node:16 node:16-alpine node:18 node:18-alpine node:20 node:20-alpine ubuntu:20.04 ubuntu:22.04	-	Removed from the Ubuntu 24.04 image due to maintenance reasons.

MikeMcC399 commented 1 month ago

@mikhailkoliada

Thanks very much for the announcement here!

Will you mark it as beta in the README?

mikhailkoliada commented 1 month ago

@MikeMcC399 yep, I forgot about it, now done :)

esaulpaugh commented 1 month ago

Run java -version java -version chmod +x ./gradlew ./gradlew build fatJar --no-daemon java -jar build/libs/headlong-cli-1.1-SNAPSHOT.jar -version shell: /usr/bin/bash -e {0} env: GRADLE_ACTION_ID: gradle/actions/wrapper-validation JAVA_HOME: /opt/hostedtoolcache/Java_Zulu_jdk/8.0.412-8/x86 JAVA_HOME_8_X86: /opt/hostedtoolcache/Java_Zulu_jdk/8.0.412-8/x86 The futex facility returned an unexpected error code. /home/runner/work/_temp/f9c9c83d-d717-4c5b-9083-6922b5f9175c.sh: line 1: 2720 Aborted (core dumped) java -version Error: Process completed with exit code 134.

andreasabel commented 1 month ago

convert (imagemagick) is no longer pre-installed.
See run https://github.com/agda/agda/actions/runs/9086817771/job/24973341288?pr=7261#step:7:105

WARNING: Unable to run the image conversion command 'convert'. 'sphinx.ext.imgconverter' requires ImageMagick by default. Ensure it is installed, or set the 'image_converter' option to a custom conversion command. Traceback: [Errno 2] No such file or directory: 'convert'

edmorley commented 1 month ago

@mikhailkoliada Thank you for the release! What is the timeline for updating the ubuntu-latest alias to default to Ubuntu 24.04 instead? (Context: We would like to upgrade to the latest Ubuntu version, however, it will save a lot of churn across many repos of replacing runs-on: ubuntu-latest with runs-on: ubuntu-24.04 if we could instead wait for the default to change.)

MikeMcC399 commented 1 month ago

@edmorley

In https://github.com/actions/runner-images/issues/9691#issuecomment-2063926929 there was mention of a date in August 2024 for GA. That might coincide with the change you are asking about.

kojoru commented 1 month ago

Are the changes marked "Removed from the Ubuntu 24.04 image due to maintenance reasons." final? Is there a particular common reason? Can we somehow vote/plead to revert any of those choices?

I expect a lot of builds with "ubuntu-latest" to break if it hits GA in that form.

mikhailkoliada commented 1 month ago

@kojoru there will definitely be changes until GA including the software list installation (for instance we are to add firefox & edge, but a bit later), but we do not guarantee we are to address all the requests, everybody is welcome to file new tools addition requests and we'll review them.

mikhailkoliada commented 1 month ago

@edmorley migrating to lates is a different process that usually takes time, 24.04 will eventually become latest after its GA

adelton commented 1 month ago

@mikhailkoliada Would you like to amend the software difference table in your first post with the information that Podman gets updated from Podman 3.4.4 in the Ubuntu 22.04 image to Podman 4.9.3 in 24.04?

sanderdelfos commented 1 month ago

@mikhailkoliada Cannot seem to use the 24.04 image:

MikeMcC399 commented 1 month ago

@sanderdelfos

I noticed also that ubuntu-24.04 does not yet appear on the Azure Pipelines > Microsoft-hosted agents list.

sanderdelfos commented 1 month ago

@MikeMcC399 True, but it is listed in this issue as ready for use and also listed here: https://github.com/actions/runner-images

dandrestor commented 1 month ago

I noticed Ubuntu2404 was not added as an option to helpers/GenerateResourcesAndImage.ps1. Are you holding off until GA or are there any plans to do that earlier?

simonmichael commented 1 month ago

Haskell tools (2204, 2404) are not mentioned in the "Software differences" list above.

And, they are older in the 2404 image. Which might be wise; although GHC 9.10 is released, most Haskell software will not build with it for a while yet.

Thanks for these, and for the new image!

pmandadkes1207 commented 1 month ago

Hello,

I want to use ubuntu-24.04 in my release pipelines but I don't see this in the "Agent Specification" drop down. How can I get it ? What is my alternative ? Currently I am able to use 24.04 in my azure-pielines.yml used in build pipelines to build image.

Thank you in advance.

f-lawe commented 1 month ago

@pmandadkes1207 It seems that Ubuntu 24.04 is not fully deployed on Azure DevOps, @sanderdelfos and I can't even use it on a YAML pipeline (but you can). Let's give it a bit more time :)

erik-bershel commented 1 month ago

Heads up! 📯 📯 📯

Couple updates regarding Ubuntu-24.04 status and updates.

The release in ADO, to our great regret, is indeed delayed, but this should not drag on for long. Please check the page describing available Microsoft Hosted agents periodically - changes will be reflected there.
The diff table includes either those packages and programs which versions we control manually, rather than providing the "latest" available version every time, or packages that are the most critical (for example, the default versions for Ruby and Python). The most complete state of the image can be found here. And please keep in mind that the situation will change quite quickly, since the availability of packages at the start of a new image changes most quickly.
We will be happy to investigate every comment and suggestion regarding the content of the new image, although we understand that we will not be able to satisfy every such request.

pmandadkes1207 commented 1 month ago

@f-lawe Sorry for the oversight on my side. I just realized that I was using 'ubuntu-latest' in azure-pipelines.yml which points to 22.04. I am gettig the same error as @sanderdelfos when I explicitly mention 24.04.

iXyles commented 1 month ago

Over the day I started to run into an issue in azure devops regarding "azure cli" after this ubuntu latest build, with the task AzureResourceManagerTemplateDeployment@3

Starting: AzureResourceManagerTemplateDeployment
==============================================================================
Task         : ARM template deployment
Description  : Deploy an Azure Resource Manager (ARM) template to all the deployment scopes
Version      : 3.238.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment
==============================================================================
ARM Service Connection deployment scope - Subscription
Checking if the following resource group exists: **REDACTED**.
Resource group exists: true.
Creating deployment parameters.
##[error]Check out the troubleshooting guide to see if your issue is addressed: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-resource-group-deployment?view=azure-devops#troubleshooting
##[error]Error: Command failed: az version
/bin/sh: 1: az: not found

Finishing: AzureResourceManagerTemplateDeployment

Is this an expected thing to be after the release? Or an unexpected breaking change?

erik-bershel commented 1 month ago

Hey @iXyles!

ubuntu-24.04 <> ubuntu-latest Your issue is related to ubuntu22.04.

iXyles commented 1 month ago

@erik-bershel Thanks for the notice! I was mixing the numbers up...

Is the additional comment in the PR above enough to get a notice on this, or should I raise a new ticket?

erik-bershel commented 1 month ago

@iXyles you may file a compatibility issue here: https://github.com/microsoft/azure-pipelines-tasks - it's a task owners repository. We won't accept this bug since base functionality of azure cli available. See there:

iXyles commented 1 month ago

Massive thanks @erik-bershel for taking the time to verify this, will look into doing that then!

EDIT: Sorry for misstag.. wrote on phone

mkniewallner commented 1 month ago

Would it be possible to have Python 3.8 as a cached version? I could understand not having 3.7, since it's end of life, but 3.8 is maintained until October 2024, so having it in the cache as well could be nice.

kasper93 commented 1 month ago

wine seems to crash with futex error

The futex facility returned an unexpected error code.
Aborted                 (core dumped) wine ./mpv.com -v --no-config

It happens also multiple times during apt-get, dpkg configuration, stage.

The futex facility returned an unexpected error code.
Aborted (core dumped)

ulysses4ever commented 1 month ago

Hey @erik-bershel. I'm one of the Cabal maintainers (the Haskell build tool included in the runners). The upgrade to GHC 9.10 was premature: nothing in the ecosystem is ready for it yet. Please, roll back to GHC 9.8 or better yet 9.6, which was declared to be a sort of an LTS by the GHC team recently.

Context. Currently, 9.10 is considered a bleeding edge. People can easily get it via ghcup (also included in the runners) but it's a bad default because, as @simonmichael mentions above: most software will fail to build with it. Cabal's own CI had a failure because we relied on the GHC from the runner for our "quick" jobs: they're supposed to run quickly, so we did't want to install any particular GHC and thought the the runner will provide a sensible default (https://github.com/haskell/cabal/pull/10026).

andreasabel commented 1 month ago

To add to the previous comment: GHC 9.10 works best with cabal 3.12 which has not been released yet.

erik-bershel commented 1 month ago

Hey @ulysses4ever and @andreasabel!

Thank you for bringing it into our attention. Lets discuss it. Currently we have next process based on the previous conversation with @andreasabel :

available_versions=$(ghcup list -t ghc -r | grep -v "prerelease" | awk '{print $2}')

# Install 2 latest Haskell Major.Minor versions
major_minor_versions=$(echo "$available_versions" | cut -d"." -f 1,2 | uniq | tail -n2)
for major_minor_version in $major_minor_versions; do
    full_version=$(echo "$available_versions" | grep "$major_minor_version." | tail -n1)
    echo "install ghc version $full_version..."
    ghcup install ghc $full_version
    ghcup set ghc $full_version
done

As I see 9.10 is declared as a latest non-prerelease version in ghcup list -t ghc -r. I think we may exclude latest if you claim that the community will only benefit from it. It's also possible to filter versions to install in a way to get recommended and latest base non-latest non-prerelease versions. Again, if it suits best the community needs. I have some vague memories that such a scheme has possible side effects due to the fact that the recommended and latest versions may be incompatible, but these may be false memories.

cc @mikhailkoliada

simonmichael commented 1 month ago

Listen to the GHC/Cabal maintainers over me, but :

+1 for waiting on GHC 9.10
Using ghcup's Recommended version would sound like a good policy.. except that's currently 9.4.8. Probably for some good reason [probably platform-specific], but many users will find ghcup's recommendation a bit too conservative.
Count me as one GHC+github actions user who enjoys having 9.8 rather than 9.6 or 9.4.

ulysses4ever commented 1 month ago

Using what ghcup deems recommended would sound like the most reasonable thing to do, except ghcup has, in my opinion, a very unintuitive and too conservative strategy for updating those. For GHC, ghcup-recommended makes some sense. For cabal, ghcup-recommended version was way out of date for a long time. So, my hunch would be to use ghcup-recommended GHC and ghcup-latest cabal (note, that generally using newer cabals with older GHCs is perfectly fine, but not vice versa). I'm fairly relaxed about what particular GHC version to get (9.4//6/8 but certainly not 9.10), though, but creating a general policy is hard...

andreasabel commented 1 month ago

@erik-bershel wrote:

available_versions=$(ghcup list -t ghc -r | grep -v "prerelease" | awk '{print $2}')

# Install 2 latest Haskell Major.Minor versions
major_minor_versions=$(echo "$available_versions" | cut -d"." -f 1,2 | uniq | tail -n2)

I think it would make sense to preinstall latest..recommended (or, if they coincide, the latest 2), and then set recommended as the default ghc. In the current situation, ghc-9.10.1 (latest), ghc-9.8.2, ghc-9.6.5 and ghc-9.4.8 (recommended) would be available and ghc would point to the last of these.

ulysses4ever commented 1 month ago

Oh, if pre-installing several versions is an option, then what Andreas suggested looks perfect to me.

albinahlback commented 1 month ago

For our Ubuntu 24.04 runners, we are getting the warning Received request to deprovision: The request was cancelled by the remote provider. followed by a cancellation of the run when it passes more than 4 minutes. Here is an example run.

I believe it started yesterday/today, and nothing noteworthy has really changed in our repository in the past couple of days.

Perhaps related to #7897.

m3dwards commented 1 month ago

I'm also getting Received request to deprovision: The request was cancelled by the remote provider on Ubuntu 24.04 jobs after 4:10 seconds.

Example

Sometimes it's failing at 4:10 but without the deprovision message and just The hosted runner encountered an error while running your job. (Error Type: Failure).

Example without deprovision message

I think there might be something resourcey going on as I've managed to get the job to run a couple of times when giving it less to do or running with less threads.

pmandadkes1207 commented 1 month ago

I am seeing another issue from azure ubuntu-latest build agent when I am building my python code from python 3.11.9 source using ubuntu 24.04 as base image.

22.30 ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them. 22.30 unknown package: 22.30 Expected sha256 xxxxx 22.30 Got yyyyy

I am not posting actual values but in my case during build. I am making python 3.11.9 as alternate install as ubuntu 24.04 by default supports only 3.12.

I wasn't seeing this issue with 20.04 which were my previous ubuntu and azure build agent versions. This is happening frequently but it's not consistent. Re-triggering the build will work.

Any idea what may be going on here ?

MichaIng commented 1 month ago

Regarding

Received request to deprovision: The request was cancelled by the remote provider

In our case, this happens when running multiple builds for multiple architectures in parallel, where APT packages are installed and upgraded within a QEMU-emulated container. To me it looks like the above warning appears for those jobs, which are currently printing/doing the

Calculating upgrade...

of APT, while installing build dependencies and upgrading the container image within the same call (apt-get dist-upgrade package1 package2 ...). All other currently running jobs of the same workflow run are then cancelled as well, no matter what they are currently doing, producing the

The hosted runner encountered an error while running your job. (Error Type: Failure).

error instead.

Surely APT calculating upgrades is CPU-intense already, probably additionally with the new APT resolver, and multiplied when being QEMU-emulated. Quite an argument for the native ARM runners, which will reduce load over hosted GitHub runners massively, once users adopt. We will ASAP 🙂.

However, the same stuff worked and works well on Ubuntu 22.04 runners, and of course a proper reaction would be to limit CPU utilisation of the runner, instead of just killing it. RAM usage btw, as indicated in one of the older linked issues, is likely not the reason, since this produces different runner-side error messages, usually just crashing whichever RAM-intense step/command currently runs and/or OOM kills, but all transparent in logs.

This, unless the VM host itself runs out of RAM. Proper reaction then would be to reduce the runners assigned RAM, or the amount of available runners. Or upgrade host hardware, of course 🙂.

EDIT: Now faced the same when checking a larger number of shell scripts with shellcheck, without any emulation.

jauderho commented 1 month ago

I am seeing the same problem as @MichaIng but it only started showing up in the last day or so. Reverting to 22.04 for now until this is fixed.

UPDATE: This now seems to be affecting ubuntu-22.04 runners too. Wondering if there is a larger issue with GitHub hosted runners.

albinahlback commented 1 month ago

For our Ubuntu 24.04 runners, we are getting the warning Received request to deprovision: The request was cancelled by the remote provider. followed by a cancellation of the run when it passes more than 4 minutes. Here is an example run.

I believe it started yesterday/today, and nothing noteworthy has really changed in our repository in the past couple of days.

Perhaps related to #7897.

Our runners now seems to be okay.

FeynmanZhou commented 1 month ago

I am a maintainer of ORAS project. Will ORAS CLI still be reserved in Ubuntu 24.04 GA runner machine?

finagolfin commented 1 month ago

@mikhailkoliada, Swift just added 24.04 support for the latest patch release, would be good to add that to this image. Unless you meant something else by "maintenance reasons" above, other than there not being official 24.04 builds till now?

Laluakku commented 3 weeks ago

![Uploading fotor_1718323087600.png…]()

MikeMcC399 commented 2 weeks ago

@mikhailkoliada

It would be great to see Firefox and Edge browsers added to the Ubuntu 24.04 beta > Browsers and Drivers section.

Is there any timeline available for this addition? I'm assuming that they are planned for the GA release of the runner in any case. Hopefully that is true?!