.Net Core installer task: Don't install if version already present on the machine

[hiyadav]

Copied issue from vsts-docs=> https://github.com/MicrosoftDocs/vsts-docs/issues/4153

Users want to ensure a version is present on the agent machine, while not reinstalling it, if it is already globally installed on the machine.

[hiyadav]

Possible solutions:

1. Update azure-pipelines-image-generation to create the markup file for each version installed on agent image.
2. Provide a checkbox for not installing the version if already present on the machine.
3. Provide a path to search for if version is installed at that location. (like /home/dotnet/)

Complications with each:

1. Users will have to global install locations as the installation path. This might lead to problems on the agents.
2. The global install location in linux machines is not set.
3. & 2 Enable Multi-Level lookup needs be ignore if any of the two cases are being used.

[MarkKharitonov]

Is there an ETA for this? We have an issue where cisco has blacklisted the IP address 199.232.36.133 and all of our pipelines are failing now because of this. If this issue was fixed, at least we could have fetched it from the cache until the cisco issue is resolved.

Here is what we get: Failed to download or parse release-index.json with error: {"errno":"ETIMEDOUT","code":"ETIMEDOUT","syscall":"connect","address":"199.232.36.133","port":443}

[StefanOssendorf]

Even the multi level lookup as documented here Dotnet core installer doc does not work as described. We are using the Windows-2019 hosted agent where according to this Hosted agent installed software readme the used runtime should already be installed and not be re-installed.

[github-actions[bot]]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

[MarkKharitonov]

Any update on this?

[mattgenious]

This seems like it would remove a lot of unnecessary queue time in pipelines and download traffic from sdk sources if applied. What is the progress on this?

[StefanOssendorf]

Any news on that?

[rmandvikar]

Any eta on this please? For me every pipeline run takes 30s to download any sdk which is waste of time.

[StefanOssendorf]

Not only a waste of time. It also uses maybe precious seconds from your free time per month.

[pulkitaggarwl]

@aaronhallberg can you have someone have a look at this ?

[StefanOssendorf]

Still nothing, huh?

[rmandvikar]

Just saying. I'm also eagerly waiting for this fix. The pipelines are such a timesink currently. It takes more time to download the sdk (30s) than it does to run all the other steps combined (<30s). An update is really appreciated.

[LennardF1989]

How can this still be open after all these years? Downloading .NET 5.x takes 30 seconds while it's already on the windows-latest image. This costs us time, and it costs Microsoft precious server resources. It works fine for the NodeTool task, why can't this just work as well?

[mattgenious]

How can this still be open after all these years? Downloading .NET 5.x takes 30 seconds while it's already on the windows-latest image. This costs us time, and it costs Microsoft precious server resources. It works fine for the NodeTool task, why can't this just work as well?

Agreed, number of builds times 30 seconds should be an absolutely insane amount of server capacity freed up if this is fixed.

[kevingermain]

I agree with @mattgenious and it surprised me as well since it's the interest of all parts

[jeroenheijmans]

I went to ask the same question on Stack Overflow before finding this GitHub thread. If things get resolved or if workarounds are found, it might be useful to update Stack Overflow as well.

[bbieniek]

I have spent the entire day today trying to figure this out. It's pretty interesting.

When you create a new pipeline and use the task:

- task: UseDotNet@2
  displayName: 'Install .NET'
  inputs:
    version: '6.0.x'
    includePreviewVersions: true

Then it will not find anything in cache. When you rerun it, it will not find anything in cache. It will however install .NET in the version 6.0.301

However, when I ran a simple script to display the dotnet version, it turns out that the dotnet version is 6.0.301.

So, to summarize - UseDotnet doesn't see that the .NET 6 is installed by default.

This bug has been opened for 2 years and each .NET install consumes 250MB and 30s of time. I wonder how many builds have been executed with this command...

I wrote a simple workaround to cache the .NET installation.

variables:
  INSTALL_DOTNET: true
  dotnet: '6.0.x'

trigger:
- master

pool:
  vmImage: ubuntu-latest

jobs:
- job: 'test'
  steps:
  - task: PowerShell@2
    displayName: Get dotnet version
    inputs: 
      targetType: inline
      script: |
        try {
          $version = (dotnet --version)
          Write-Host "Found version: $version"        
          $expectedVersion = '$(dotnet)'
          Write-Host "Expected version: $expectedVersion"
          if ($version) {        
            if ($version -like $expectedVersion.Replace('x', '*')) 
            {
                    Write-Host "The installed version $version matches the expected dotnet version $expectedVersion"
                    Write-Host "##vso[task.setVariable variable=INSTALL_DOTNET;]$false"
            }
            else {
              Write-Host "The installed version $version doesnt match the expected dotnet version $expectedVersion"
            }
          }
          else {
            Write-Host "Dotnet not found"
          }        
        }
        catch {
          Write-Host "Dotnet not found"
        }

  - task: Cache@2
    inputs:
      key: '"dotnet" | "$(Agent.OS)" | "$(dotnet)"'
      path: $(Agent.ToolsDirectory)/dotnet
      cacheHitVar: CACHE_RESTORED
    displayName: Cache Dotnet packages
    condition: eq(variables['INSTALL_DOTNET'], 'True')

  - task: UseDotNet@2
    displayName: 'Install .NET'
    inputs:
      version: $(dotnet)
      includePreviewVersions: true
      installationPath: $(Agent.ToolsDirectory)/dotnet
    condition: eq(variables['INSTALL_DOTNET'], 'True')

All that is needed to solve this issue is to check in those if statements: https://github.com/microsoft/azure-pipelines-tasks/blob/878d96966537e41fbc80ce6b02c3bee4cb7b7a1b/Tasks/UseDotNetV2/versioninstaller.ts#L102

The folder of /usr/share/dotnet/sdk:

[rmandvikar]

@pulkitaggarwl Any update on this please? Quite a few people, including me, are waiting on this. It's confusing in the current state, as either the doc should be fixed, or the bug acknowledged and fixed. I read this thread, and can't tell if this is a bug or expected behavior. Please clarify.

[KerbalSpike]

It is really mind-boggling, that there has not been any fix for this. I mean I could understand it if it would only be affecting and annoying users, but this must be a pretty big resource hog for Microsoft's infrastructure, assuming that there must be thousands of pipelines running with this task each and every day...

[weshaggard]

@bishal-pdMSFT are there any plans to add this support? It is definitely an issue that lots of folks have to deal with and potentially make workarounds for.

[doggy8088]

I have a workaround for this issue.

Don't use the UseDotNet@2 task, just create a global.json file before your dotnet build by using Command Line (CMD or Bash).

Don't use dotnet new globaljson --sdk-version 6.0.100 --roll-forward latestFeature for this!

Here is an example:

Command Prompt

echo {"sdk":{"version":"6.0.100","rollForward":"latestFeature","allowPrerelease":false}} > global.json

Bash

echo '{"sdk":{"version":"6.0.100","rollForward":"latestFeature","allowPrerelease":false}}' > global.json

[npbenjohnson]

Setting a global.json works if the version you want is definitely installed, but it will break when not installed, this workaround fixes the task's cacheing: https://github.com/actions/runner-images/issues/4884#issuecomment-1013047819

Before seeing that fix I made one that uses the dotnet cli and bash against global.json. This won't support rollForward behavior, but we always keep our CI at the exact sdk version of Visual Studio so dotnet format works consistently between CI and dev.

      # Use this task to check if version is installed or not and emit variable
      - bash: |
          version=$(jq -r .sdk.version global.json)
          echo "global.json version - $version"
          if dotnet --list-sdks | grep -q $version;
          then
            echo "Sdk is installed"
            echo "##vso[task.setvariable variable=sdkVersionIsPresent;isOutput=true]true"
          else
            echo "Sdk is not installed"
            echo "##vso[task.setvariable variable=sdkVersionIsPresent;isOutput=true]false"
          fi
        displayName: Check Sdk
        workingDirectory: $(System.DefaultWorkingDirectory)
        name: InitializeEnvironment

      # Use this task from within the InitializeEnvironment job (if needed)
      - task: UseDotNet@2
        displayName: Set Sdk (Not pre-installed)
        condition: and(succeeded(), ne(variables['InitializeEnvironment.sdkVersionIsPresent'], true))
        inputs: { packageType: sdk, useGlobalJson: true }

     # Use this job variable for jobs in same stage
    variables:
      sdkVersionIsPresent: $[ dependencies.[Name Of Job Containing InitializeEnvironment].outputs['InitializeEnvironment.sdkVersionIsPresent'] ]

      # Use this job variable for jobs in other stages
    variables:
      sdkVersionIsPresent: $[ stageDependencies.[Name Of Stage Containing InitializeEnvironment].[Name Of Job Containing InitializeEnvironment].outputs['InitializeEnvironment.sdkVersionIsPresent'] ]

      # Use this task from within jobs that has the variable above
      - task: UseDotNet@2
        displayName: Set Sdk (Not pre-installed)
        condition: and(succeeded(), ne(variables.sdkVersionIsPresent, true))
        inputs: { packageType: sdk, useGlobalJson: true }

[github-actions[bot]]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

[mattgenious]

This seems to work for me with .net 8 sdk being preinstalled and not having to install it, is that the same experience others have?

[IvanAlekseev]

It is still the same for us. Agent info:

Operating System
Ubuntu
22.04.3
LTS
Runner Image
Image: ubuntu-22.04
Version: 20240107.1.0
Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240107.1/images/ubuntu/Ubuntu2204-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240107.1
Runner Image Provisioner
2.0.321.1
Current image version: '20240107.1.0'

Task

          - task: UseDotNet@2
            inputs:
              packageType: sdk
              version: 8.0.x
              displayName: dotnet install

Logs

##[section]Starting: UseDotNet
==============================================================================
Task         : Use .NET Core
Description  : Acquires a specific version of the .NET Core SDK from the internet or the local cache and adds it to the PATH. Use this task to change the version of .NET Core used in subsequent tasks. Additionally provides proxy support.
Version      : 2.232.0
Author       : Microsoft Corporation
Help         : https://aka.ms/AA4xgy0
==============================================================================
Tool to install: .NET Core sdk version 8.0.x.
Found version 8.0.100 in channel 8.0 for user specified version spec: 8.0.x
get-os-distro: Error: Distribution specific OS name and version could not be detected: UName = Linux
Version 8.0.100 was not found in cache.
Getting URL to download .NET Core sdk version: 8.0.100.
Detecting OS platform to find correct download package for the OS.
[command]/home/vsts/work/_tasks/UseDotNet_b0ce7256-7898-45d3-9cb5-176b752bfea6/2.232.0/externals/get-os-distro.sh
Primary:linux-x64
Legacy:-x64
Detected platform (Primary): linux-x64
Detected platform (Legacy): -x64
Downloading: https://download.visualstudio.microsoft.com/download/pr/5226a5fa-8c0b-474f-b79a-8984ad7c5beb/3113ccbf789c9fd29972835f0f334b7a/dotnet-sdk-8.0.100-linux-x64.tar.gz
Extracting downloaded package /home/vsts/work/_temp/7e80ed41-c353-4e64-873e-f580fa4c8360.
Extracting archive
[command]/usr/bin/tar xC /home/vsts/work/_temp/cea2d5f9-0ff5-46c5-86b6-7da4a5a77cd3 -f /home/vsts/work/_temp/7e80ed41-c353-4e64-873e-f580fa4c8360
Successfully installed .NET Core sdk version 8.0.100.
Creating global tool path and pre-pending to PATH.
##[section]Finishing: UseDotNet

[mattgenious]

Oh right, I just deleted the install step but then obviously that introduces the uncertainty of not guaranteeing the sdk version being installed, makes sense.

[StefanOssendorf]

Still nothing? This is now nearly 5 years old and could have saved a lot of time and resources...

[Varorbc]

any update?