search

actions / runner-images

GitHub Actions runner images
MIT License
9.17k stars 2.84k forks source link

Add .NET 6 SDK #4424

Closed akyamada-ms closed 2 years ago

akyamada-ms commented 2 years ago

Tool name

.NET 6 SDK

Tool license

N/A

Add or update?

Desired version

N/A

Approximate size

No response

Brief description of tool

I’m support engineer in Japan. One of my customers is using Ubuntu latest OS image for Microsoft-hosted agents in Azure DevOps Services. Does not have .NET 6 installed in Ubuntu latest OS image, so he adds the following definition to Pipeline and installs it every time.

steps:

I think you already have a plan, but I would like to request to include .NET 6 in the OS image.

Also, I would appreciate it if you could let me know when the OS image will include .NET 6 and if there is any publicly available information about future plans.

URL for tool's homepage

https://github.com/dotnet/core/tree/main/release-notes/6.0

Provide a basic test case to validate the tool's functionality.

No response

Virtual environments affected

Can this tool be installed during the build?

steps:
- task: UseDotNet@2
displayName: 'Use .NET Core sdk'
inputs:
packageType: sdk
version: 6.0.100-rc.2.21505.57
installationPath: $(Agent.ToolsDirectory)/dotnet

Tool installation time in runtime

N/A

Are you willing to submit a PR?

No response

al-cheb commented 2 years ago

Hey, @akyamada-ms .NET 6 will launch at .NET Conf November 9-11(https://devblogs.microsoft.com/dotnet/announcing-net-6-release-candidate-2/) and take 1 or 2 weeks to new rollout.

miketimofeev commented 2 years ago

Hi @akyamada-ms! How long does it take to install .Net 6 in runtime? We are thinking of changing the policy and do not pre-install .NET on the images due to maintenance and security concerns.

akyamada-ms commented 2 years ago

Thank you, @al-cheb @miketimofeev I tried it five times with ubuntu-latest and it took an average of 8.6 seconds. It looks like there is no problem with the Pipeline execution time.

epvanhouten commented 2 years ago

@miketimofeev is the guidance going forward to rely on UseDotNet@2 to get dotnet onto the agent?

miketimofeev commented 2 years ago

@epvanhouten using setup- actions or Use tasks is a recommended way to install the desired version.

mikeblakeuk commented 2 years ago

Make sure you force the base path https://github.com/MicrosoftDocs/azure-devops-docs/issues/9690

AraHaan commented 2 years ago

.NET 6 has been out today (a day early), I can manually install it today however when invoking an .NET global tool it will fail on me saying that the Microsoft.NETCore.App version 6.0.0 was not found in /usr/share/dotnet/shared/.

I guess I could have it fallback to sudo dotnet [global tool command] and see if that works though.

AraHaan commented 2 years ago

Ok I found a workaround for this:

use the following locally and commit these to repository:

dotnet new tool-manifest

from repository root.

Then install all the tools you need without -g or --global in the command so they get added to that manifest.

Then in the github actions workflows run dotnet tool restore and then run the commands like normal and they will run exactly the way they was made to even if they use the .NET 6 runtime itself.

mikeblakeuk commented 2 years ago

You are hitting the bug I had https://github.com/domaindrivendev/Swashbuckle.AspNetCore/issues/1906 I'm just going to wait this time. Note; .net 6.0 is on the windows-2022 builds, just not ubuntu yet

zcsizmadia commented 2 years ago

@AraHaan

This will install the latest .NET 3.1, 5.0 and 6.0 in the github action. Adding this will make the action future proof, in case Github decides to remove the dotnet SKDs from the latest images. @miketimofeev mentioned earlier, that it is under consideration. Btw each SDK install takes 6-7s.

    # Install .NET SDKs
    - name: Setup .NET Core 3.1
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 3.1.x
    - name: Setup .NET 5.0
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 5.0.x
    - name: Setup .NET 6.0
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 6.0.x
LanceMcCarthy commented 2 years ago

Just to add, this request is more than just installing NET6 RTM to the runner. We also need VS 2022 Current and it's NET6.0 based tooling (like msbuild).

miketimofeev commented 2 years ago

@LanceMcCarthy the VS2022 stable image with .NET 6.0 is currently rolling out and will be generally available tomorrow.

al-cheb commented 2 years ago

@LanceMcCarthy , Windows Server 2022 with VS2022 has been deployed.

AraHaan commented 2 years ago

Actually I replaced the setup-dotnet github action, but I figured out a solution to the .NET Core global tools issue.

https://github.com/Elskom/runtime/blob/main/.github/workflows/dotnetcore-build.yml#L8-L32

And then at the root of the repository I did:

dotnet new tool-manifest

And then I run the tool install commands without the -g or --global flags to add it to the tool manifest so the CI can restore it.

After that I use dotnet tool run so that way the installed .NET 6 SDK runs it and then it honors DOTNET_ROOT in this case since it's an framework dependent tool package.

miketimofeev commented 2 years ago

As we decided not to add .NET for all the images (at least for now) I'm going to close the issue. Please feel free to contact us if you have any concerns. Thank you!

mikeblakeuk commented 2 years ago

When will the Ubuntu 20.04 DevOps agent have .NET 6.0 pre installed? (in the same style that it has 5.0)

miketimofeev commented 2 years ago

@mikeblakeuk please consider using the task https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/tool/dotnet-core-tool-installer?view=azure-devops Please provide a reason if it's not possible for you.

AtOMiCNebula commented 2 years ago

As we decided not to add .NET for all the images (at least for now) I'm going to close the issue. Please feel free to contact us if you have any concerns. Thank you!

@miketimofeev, can you elaborate on how this decision was reached? The answer isn't obvious to me (and a few others, given the downvotes on your comment). .NET 6 is ultimately getting installed in the windows-2022 images, likely from the VS2022 installation process (it's not listed in the dotnet toolset section for either -2019 or -2022).

What will happen to the -2019 image when .NET 5 reaches EoL in May 2022, or when .NET Core 3.1 reaches EoL in December 2022?

AraHaan commented 2 years ago

I think they uninstall the EOL versions of .NET if they are not a critical part of Visual Studio.

AtOMiCNebula commented 2 years ago

Will we have no pre-installed SDKs then? Is that a happy position? It feels inconsistent to me, especially if getting .NET 6 onto the windows-2022 agents is a side-effect and not necessarily an intended effect.

AraHaan commented 2 years ago

In the mean time I would suggest using the github action I made and follow the steps I posted above using my workflow as an example. This should work 100% of all needed usage of the .NET 6 SDK.

Meertman commented 2 years ago

The proposed work-around of specifying the .NET SDK version to use, does not work when executing unit and integration tests using multiple target frameworks. (For example when building a library that targets ASP.NET Core 3.1, ASP.NET 5 and ASP.NET 6)

This will ensure that the tests for .NET Core 3.1 and .NET 5 fail with the following message:

ASP.NET Core 3.1: The framework 'Microsoft.AspNetCore.App', version '3.1.0' (x64) was not found.

ASP.NET 5 The framework 'Microsoft.AspNetCore.App', version '5.0.0' (x64) was not found.

AraHaan commented 2 years ago

The proposed work-around of specifying the .NET SDK version to use, does not work when executing unit and integration tests using multiple target frameworks. (For example when building a library that targets ASP.NET Core 3.1, ASP.NET 5 and ASP.NET 6)

This will ensure that the tests for .NET Core 3.1 and .NET 5 fail with the following message:

ASP.NET Core 3.1: The framework 'Microsoft.AspNetCore.App', version '3.1.0' (x64) was not found.

* The following frameworks were found:
  6.0.0 at [/opt/hostedtoolcache/dotnet/shared/Microsoft.AspNetCore.App]

ASP.NET 5 The framework 'Microsoft.AspNetCore.App', version '5.0.0' (x64) was not found.

* The following frameworks were found:
  6.0.0 at [/opt/hostedtoolcache/dotnet/shared/Microsoft.AspNetCore.App]

That is because when you specified .NET 6 you did not install the other SDKs before that in order to install their runtimes for those other TFMs.

Meertman commented 2 years ago

@AraHaan, that is correct, so the work-around does not work in this case, or am I missing something?

Or can you specify multiple SDK's?

maxkoshevoi commented 2 years ago

Or can you specify multiple SDK's?

Just execute setup-... task multiple times with different SDK versions

zcsizmadia commented 2 years ago

@Meertman https://github.com/actions/setup-dotnet/pull/240 just got merged. It got much easier to deploy multiple frameworks. Check https://github.com/actions/setup-dotnet for more examples:

steps:
- name: Setup dotnet
- uses: actions/checkout@v2
- uses: actions/setup-dotnet@v1
  with:
    dotnet-version: | 
      3.1.x
      5.0.x
      6.0.x
- run: dotnet build <my project>
AraHaan commented 2 years ago

I guess I could update my setup-latest-dotnet action with the ability to specify multiple .NET versions as well.

GGG-KILLER commented 2 years ago

@Meertman actions/setup-dotnet#240 just got merged. It got much easier to deploy multiple frameworks. Check https://github.com/actions/setup-dotnet for more examples:

steps:
- name: Setup dotnet
- uses: actions/checkout@v2
- uses: actions/setup-dotnet@v1
  with:
    dotnet-version: | 
      3.1.x
      5.0.x
      6.0.x
- run: dotnet build <my project>

Just a reminder that the v1 tag hasn't been updated to it yet, so the snippet you posted isn't working yet

LanceMcCarthy commented 2 years ago

Indeed, the latest published version (v1.8.3) was released on Sept 14th 2021. Additionally, I do not see any pending builds in the release workflow https://github.com/actions/setup-dotnet/actions/workflows/release-new-action-version.yml

So, the current readme is a bit misleading. I've opened an issue to ask them to push out a new release https://github.com/actions/setup-dotnet/issues/249

LanceMcCarthy commented 2 years ago

FYI - setup-dotnet v1.9 is now available https://github.com/actions/setup-dotnet/releases/tag/v1.9.0.

You can replace your consecutive individual dotnet-install steps (or custom install-dotnet.sh scripts), with the following:

- uses: actions/setup-dotnet@v1.9.0
  with:
    dotnet-version: | 
      3.1.x
      5.0.x
      6.0.x

Edit: Updated example to prevent accidental copy/paste, in which you'll run into the implicit version issue @GGG-KILLER mentions.

GGG-KILLER commented 2 years ago

FYI - setup-dotnet v1.9 is now available https://github.com/actions/setup-dotnet/releases/tag/v1.9.0.

You can replace your consecutive individual dotnet-install steps (or custom install-dotnet.sh scripts), with the following:

- uses: actions/setup-dotnet@v1
  with:
    dotnet-version: | 
      3.1.x
      5.0.x
      6.0.x

The workflow run to update the v1 tag is still pending, so if you want to use this, you'll have to use actions/setup-dotnet@v1.9.0 until the workflow has finished running

mikeblakeuk commented 2 years ago

What should we use for Azure DevOps when using latest ubuntu?

LanceMcCarthy commented 2 years ago

What should we use for Azure DevOps when using latest ubuntu?

@mikeblakeuk In ADO, you can add a step for each version of .NET you want installed. For example, here I am adding 6.0 at the top of my stack:

image

If you're using YAML instead of classic:

- task: UseDotNet@2
  inputs:
    packageType: 'sdk'
    version: '6.0.x'

If you need more than one, add an additional step for each version you need.

weshaggard commented 2 years ago

please consider using the task https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/tool/dotnet-core-tool-installer?view=azure-devops Please provide a reason if it's not possible for you.

While we can use the task it is far less reliable then having it already on the agent. We just switched to .NET 6.0 SDK a few days ago and already seeing random issues like:

gzip: stdin: unexpected end of file
/usr/bin/tar: Unexpected EOF in archive
/usr/bin/tar: Unexpected EOF in archive
/usr/bin/tar: Error is not recoverable: exiting now
##[error]Failed while installing version: 6.0.100 at path: /opt/hostedtoolcache/dotnet with error: Failed while extracting downloaded package with error: Error: The process '/usr/bin/tar' failed with exit code 2
LanceMcCarthy commented 2 years ago

@weshaggard That end of file error is something that happens when an archive is empty or corrupt (i.e. a bad download)

You'll want to use the "Use .NET" Task instead of manually installing stuff. Mainly, it's more reliable, but much neater 😎 It will detect the OS, properly install it in the correct location and will update PATH.

In GitHub Actions

- uses: actions/setup-dotnet@v1
  with:
    dotnet-version: '6.0.x'

In Azure DevOps

- task: UseDotNet@2
  inputs:
    packageType: 'sdk'
    version: '6.0.x'

If you're still using ADO Classic pipeline, see my previous reply for a screenshot.

weshaggard commented 2 years ago

We are using the DevOps task I just didn't post the full log:

Starting: Use .NET Core SDK
==============================================================================
Task         : Use .NET Core
Description  : Acquires a specific version of the .NET Core SDK from the internet or the local cache and adds it to the PATH. Use this task to change the version of .NET Core used in subsequent tasks. Additionally provides proxy support.
Version      : 2.195.0
Author       : Microsoft Corporation
Help         : https://aka.ms/AA4xgy0
==============================================================================
get-os-distro: Error: Distribution specific OS name and version could not be detected: UName = Linux
Found version 6.0.100 in channel 6.0 for user specified version spec: 6.0.100
Getting URL to download .NET Core sdk version: 6.0.100.
Detecting OS platform to find correct download package for the OS.
/mnt/vss/_work/_tasks/UseDotNet_b0ce7256-7898-45d3-9cb5-176b752bfea6/2.195.0/externals/get-os-distro.sh
Primary:linux-x64
Legacy:-x64
Detected platform (Primary): linux-x64
Detected platform (Legacy): -x64
Version 6.0.100 was not found in cache.
Downloading: https://download.visualstudio.microsoft.com/download/pr/17b6759f-1af0-41bc-ab12-209ba0377779/e8d02195dbf1434b940e0f05ae086453/dotnet-sdk-6.0.100-linux-x64.tar.gz
Extracting downloaded package /mnt/vss/_work/_temp/0d61359d-0e06-4967-81ce-813ed3b62856.
Extracting archive
/usr/bin/tar xC /mnt/vss/_work/_temp/72b79f24-d279-4669-a01b-d01991e9f316 -f /mnt/vss/_work/_temp/0d61359d-0e06-4967-81ce-813ed3b62856

gzip: stdin: unexpected end of file
/usr/bin/tar: Unexpected EOF in archive
/usr/bin/tar: Unexpected EOF in archive
/usr/bin/tar: Error is not recoverable: exiting now
##[error]Failed while installing version: 6.0.100 at path: /opt/hostedtoolcache/dotnet with error: Failed while extracting downloaded package with error: Error: The process '/usr/bin/tar' failed with exit code 2
Finishing: Use .NET Core SDK
lindexi commented 2 years ago

Any update?

jetersen commented 2 years ago

@miketimofeev any chance this can be reconsidered? On Windows VM it is at least 30 seconds slower on average when compared to Ubuntu VM for GitHub runner. Most likely due to network issues reported in #3577

AraHaan commented 2 years ago

I am working on something with my github action to do something about this.

I plan to do stuff like allowing to pass in a list of .NET versions to install.

However it would be nice to implement caching for those who specify explicit versions and does not wildcard a specific version (for the .NET install scripts to install).

miketimofeev commented 2 years ago

@jetersen we will discuss this internally. There is a chance we will pre-install only the latest available 6.* version on the image

AraHaan commented 2 years ago

So basically there is a chance that it may be 6.0.102 or 6.0.200.

jchannon commented 2 years ago

I've just hit this issue having moved all our code to .NET 6, I have a separate task that uses ubuntu-latest only to find that MS have decided not to include .NET 6 and now our deployment is broken. (https://github.com/actions/virtual-environments/issues/4424#issuecomment-967012616)

What was the reasoning by not including .NET 6 on ubuntu images, this is nuts!

akrock commented 2 years ago

@miketimofeev You posted a month ago that this would be under discussion. How is that discussion coming???

The rest of us out here in the real world are still pretty shocked that the windows hosted agents come with NET 6, and the linux ones do not. So much for an equitable approach for dotnet 5+ ...

The sheer cost in lost engineering time that this questionable decision is going to cause is only going to grow over time as more teams try to adopt NET 6 only to find it doesn't work out of the box if they run linux hosted agents.

[EDIT] Actual Repercussions

For what it's worth this was a minor annoyance for us, it sucked up about 3 hours of time in total from discovery to resolution. We have zero .NET 6 code at the moment, however one of our engineers attempted to use file-scoped namespaces while using Visual Studio 2022 and we caught this when a PR was submitted. It 100% builds locally while the project is targeting NET 5 for anyone who has Visual Studio 2022 installed, or the NET 6 SDK and the latest Rider IDE. I would assume people using VS Code would also be able to locally compile files with file-scoped namespaces but I have not tested this personally. No one likes when code compiles locally but fails at the build server :-1:.

We have 62 pipelines defined in Azure DevOps. Of those 62 pipelines - 47 needed to be updated from using the pre-installed SDKs (that we have used historically since we migrated to AZP several years ago) to adding two actions.

image

We are fortunate for several things... one we just late last year migrated ALL of our net core code to net5. Up until very recently we would have needed to add 4 (or more) different SDK versions because adding just 6.0 makes everything else break.

Secondly that we have built our own functionality to "template" out the Pipelines, such that we can make a change to one place and copy it to all the other defined pipelines. Otherwise updating these 47 legacy (non-yaml ... i know ... 🤮 ) pipelines would have been a hurdle that we probably would not have crossed without major prodding.

🍿 🍿 This is gonna be fun once people are forced off NET 5 - May 8, 2022 🍿 🍿

jetersen commented 2 years ago

🤯 I was not aware of the fact that Windows had .NET 6 installed and the active decision is to NOT install it on ubuntu.

But it clearly shown here: https://github.com/actions/virtual-environments/blob/main/images/win/Windows2022-Readme.md#net-core-sdk

Though seems the script for this is broken.

The toolset is defined here: https://github.com/actions/virtual-environments/blob/8e0e0d68b96a85314b5e16a6a5896244672a83f6/images/win/toolsets/toolset-2022.json#L281-L287

Seems the powershell script ignores that based on the software report. https://github.com/actions/virtual-environments/blob/8e0e0d68b96a85314b5e16a6a5896244672a83f6/images/win/scripts/Installers/Install-DotnetSDK.ps1#L91-L100

I guess this is due to Visual Studio Installer having a field day installing varies workloads 😅

jetersen commented 2 years ago

@miketimofeev

miketimofeev (Mikhail Timofeev) on Nov 8, 2021 We are thinking of changing the policy and do not pre-install .NET on the images due to maintenance and security concerns.

If the argument is maintenance and security why is it installed on Windows machines (with a variety of patches: 3.1.120 3.1.202 3.1.302 3.1.416 5.0.104 5.0.210 5.0.303 5.0.404 6.0.101) To NOT have a single 6.0.101 installed on ubuntu/macos images?

Again if the argument is maintenance and security simply give us the latest patched .NET LTS (Active) and Current? Much like Node, Python, Java, Ruby, Go, PHP, Xcode, Xamarin, Android SDK. For now this would mean:

according to https://dotnet.microsoft.com/en-us/platform/support/policy/dotnet-core#lifecycle

I think it would be fine to support the lifecycle and for anything else or a if you need a specific patch or minor version of .NET use setup-dotnet action.

Maybe, it is just me. Sort of mind-boggling that a Microsoft/GitHub staff arguing for maintenance and security concerns for their own open source product. Yet happily installs varies Android SDK, python versions etc. etc. pre installed on exact same hosted image.

WHAT? REALLY?

miketimofeev commented 2 years ago

@jetersen @akrock .Net 6 on Windows machines is bundled with VS2022. I'm afraid we don't have much progress regarding adding it to all the other images and I can't share the details. I will update the issue if something changes. Sorry for the inconvenience.

shoter commented 2 years ago

I think it should be listed higher on list of Microsoft priorities to bring it up because in less than 4 months you are dropping support of .NET 5 whereas .NET 6 is not yet fully ready as we see.

Or increase support time for .NET 5 if you are not yet ready for whatever "security" reasons you have.

Steve-Glass commented 2 years ago

👋 Hi everyone! I’m Steve, the Product Manager working in the virtual environments domain. We’ve been actively reviewing feedback from the community on this issue and have been internally discussing the addition of the .NET 6 SDK to the Ubuntu image. The community feedback, internal discussions, and review of security and maintenance concerns/requirements have led us to the following solution:

In regards to ETA, I’ll be working with the team to prioritize software updates and we will communicate via this issue. Please let us know if you have any additional questions and thank you for voicing concerns with our team!

jetersen commented 2 years ago

@Steve-Glass Thanks for reaching a decision. Latest SDK is a good compromise which I hope the majority will find acceptable. For my own case that is very true 😊

zcsizmadia commented 2 years ago
  • We will keep the Ubuntu image updated with the latest .NET SDK by default.

@Steve-Glass It is true for all supported SDKs, right, and not just 6.0? So the latest 3.1 5.0 and 6.0 SDKs would be included.