`DotNetCoreCLIV2` should respect `rollForward` in `global.json`

EricStG commented 4 years ago

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Feature

Enter Task Name: DotNetCoreCLIV2

Environment

Server - Azure Pipelines or TFS on-premises?

Using Azure Pipelines but this isn't environment specific

Agent - Hosted or Private:

Using self hosted agents but this isn't environment specific

Issue Description

When using DotNetCoreCLIV2 and pulling the information from a global.json file, it's not considering [rollForward[(https://docs.microsoft.com/en-us/dotnet/core/tools/global-json?tabs=netcore3x). Example, if specifying "version": "3.1.100" with "rollForward": "latestPatch", it should be able to install 3.1.106 (or whichever is the latest at that time)

It might be possible to do something similar using 3.1.x as a version number, but that doesn't seem to be valid within global.json.

Task logs

Starting: UseDotNet
==============================================================================
Task         : Use .NET Core
Description  : Acquires a specific version of the .NET Core SDK from the internet or the local cache and adds it to the PATH. Use this task to change the version of .NET Core used in subsequent tasks. Additionally provides proxy support.
Version      : 2.171.0
Author       : Microsoft Corporation
Help         : https://aka.ms/AA4xgy0
==============================================================================
Found version 3.1.100 in channel 3.1 for user specified version spec: 3.1.100
Getting URL to download .NET Core sdk version: 3.1.100.
Detecting OS platform to find correct download package for the OS.
/home/vsts/work/_tasks/UseDotNet_b0ce7256-7898-45d3-9cb5-176b752bfea6/2.171.0/externals/get-os-distro.sh
Primary:linux-x64
Legacy:ubuntu.18.04-x64
Detected platform (Primary): linux-x64
Detected platform (Legacy): ubuntu.18.04-x64
Version 3.1.100 was not found in cache.
Downloading: https://download.visualstudio.microsoft.com/download/pr/d731f991-8e68-4c7c-8ea0-fad5605b077a/49497b5420eecbd905158d86d738af64/dotnet-sdk-3.1.100-linux-x64.tar.gz
Extracting downloaded package /home/vsts/work/_temp/4f922889-2d5d-47d8-9904-cfaaebd08953.
Extracting archive
/bin/tar xzC /home/vsts/work/_temp/c44417c1-deba-40ec-9856-2fa4c4a3c1f8 -f /home/vsts/work/_temp/4f922889-2d5d-47d8-9904-cfaaebd08953
Successfully installed .NET Core sdk version 3.1.100.
Creating global tool path and pre-pending to PATH.
Finishing: UseDotNet

logs_32.zip

Troubleshooting

Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting

Error logs

[Insert error from the logs here for a quick overview]

bishal-pdMSFT commented 3 years ago

We are not planning to make this enhancement. A workaround exists to use a well tested complete version in global.json.

twurm commented 3 years ago

@bishal-pdMSFT can you point me in the direction of the workaround, or did you just mean the full version you want to use? I would think that keeping feature parity with what happens in the local developer experience and that of Azure DevOps would make everyone's life easier.

It might be helpful to update the documentation for this task to explain that it doesn't respect the rollforward settings.

arthurrump commented 2 years ago

I have to say I was rather surprised to discover that DevOps was installing version 5.0.400, even though there have been patch releases since and I had set rollForward to latestFeature. Given that the flag is called useGlobalJson, I had expected the version resolution to behave like specified in the global.json docs.

If I understand correctly, there are two "workarounds" to run on e.g. the latest patch release of an sdk:

Manually roll forward the version set in global.json every time there is an update. I think this is undesirable, and given that the rollForward option exists, I'm probably not the only one. When using rollForward you specify the minimum version that should be used on the project, and allow for higher versions to be used if available. Setting the version to the latest patch release also forces every developer working on the project to install this latest patch version, which again is not desirable per se.
Use the version specification field with e.g. 3.1.x in the task instead of using global.json. The main disadvantage is that now there are two places where the required .NET sdk version is specified: once for CI in the CI configuration and once for all other environments in global.json. That's not great, and given that the useGlobalJson option exists, I'm sure I'm not the only one who thinks so.

@bishal-pdMSFT Please reconsider if a workaround is really good enough here. For me it defeats the purpose of having the useGlobalJson feature in the first place.

sadjadbp commented 2 years ago

@bishal-pdMSFT the answer really does NOT make sense, why are you forcing us to put sdk in 2 places, basically what's the purpose of supporting global.json if you are not following the dotnet rules!

Herve-M commented 1 year ago

@bishal-pdMSFT Would it be possible to share how to A workaround exists to use a well tested complete version in global.json. ?

justinswork commented 1 year ago

@bishal-pdMSFT Would it be possible to share how to A workaround exists to use a well tested complete version in global.json. ?

Since Microsoft doesn't appear to be in too much of a hurry to help with this issue, I'll share the workaround I'm using.

Overview

Essentially, I can't trust the UseDotNet task (or any subsequent DotNet tasks) to do what I want unless I have a global.json that specifies exactly which version to use. This is a problem because If I have a global.json checked into the repository with a specific version (with no rollForward), it will no longer be able to build on my local machine as soon as I get a Windows update that replaces that version with the latest patch. Meaning I will need to constantly be updating that file on the repository. But again, the UseDotNet task doesn't work properly without the global.json. It will download the specified version to the build agent, but future dotnet tasks will not use that version if there is a newer version of DotNet installed.

I have observed this incorrect behavior multiple times and tried numerous workarounds and this is the best solution I have come up.

Solution

Remove the global.json file from my repository
Only build the global.json on demand while running the pipeline, before I do any other DotNet tasks (including UseDotNet). (For those who may be unfamiliar, this is how I build the global.json in the pipeline https://learn.microsoft.com/en-us/dotnet/core/tools/global-json#globaljson-and-the-net-cli)
Set useGlobalJson to true in the UseDotNet task https://learn.microsoft.com/en-gb/azure/devops/pipelines/tasks/reference/use-dotnet-v2?view=azure-pipelines&viewFallbackFrom=azure-devops
This newly created global.json file will both ensure that my UseDotNet task installs the version specified, and that future dotnet tasks will use that version as well.
Since I don't have a global.json in the repository anymore, my local machine will just build with whatever version is currently installed. This gives me the freedom to determine when I want to update the version that my build agent uses without breaking builds on local machines.

It's unfortunate that this workaround needs to exist, given how flexible the global.json was created to be, and I would hope that Microsoft does consider improving this in the near future but until then, this is what I will be doing. Hopefully this will help others who have been struggling with the same issues.

eakoning commented 1 year ago

@justinswork I noticed similar behaviour when comparing results between a self hosted agent and a Microsoft hosted agent. The Microsoft agent properly respected my UseDotNet task, downloaded the SDK, added it to PATH and subsequent tasks were using that exact SDK version.

When checking the logs of the self hosted agents however (which are using a limited permissive service account) it seemed the UseDotNet task was working correctly, even telling me it had appended stuff to the PATH, but I was still seeing subsequent dotnet related tasks using the latest installed SDK.

Now things seem to be working fine by removing the UseDotNet task alltogether and using global.json with the following contents:

{
  "sdk": {
    "version": "6.0.0",
    "rollForward": "latestFeature",
    "allowPrerelease": false
  }
}

The subsequent tasks are now using SDK v6.0.413

microsoft / azure-pipelines-tasks