search

dotnet / dotnet-docker

Docker images for .NET and the .NET Tools.
https://hub.docker.com/_/microsoft-dotnet
MIT License
4.49k stars 1.94k forks source link

"dotnet pack" does not work using the SDK 8.0 container in Azure DevOps pipeline #5745

Closed rklec closed 3 months ago

rklec commented 3 months ago

Describe the Bug

dotnet pack fails inside of the conmtainer with wild errors like:

Assembly loaded during Evaluation: System.Reflection.Metadata, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (location: /usr/share/dotnet/shared/Microsoft.NETCore.App/8.0.7/System.Reflection.Metadata.dll, MVID: 6a8d52f3-3d62-4da3-8b4f-a8dd17a8d101, AssemblyLoadContext: Default)
       The SDK "Microsoft.NET.Sdk" was successfully resolved by the "DefaultSdkResolver" resolver to location "/usr/share/dotnet/sdk/8.0.303/Sdks/Microsoft.NET.Sdk/Sdk" and version "".
       Assembly loaded during Evaluation: Microsoft.NET.StringTools, Version=1.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (location: /usr/share/dotnet/sdk/8.0.303/Microsoft.NET.StringTools.dll, MVID: a8215d72-494e-46bc-80f3-3ce7f496c99c, AssemblyLoadContext: Default)       
       // later

The "DefaultSdkResolver" resolver attempted to resolve the SDK "Microsoft.NET.SDK.WorkloadAutoImportPropsLocator".
       Warnings: null
       Errors: MSB4276: The default SDK resolver failed to resolve SDK "Microsoft.NET.SDK.WorkloadAutoImportPropsLocator" because directory "/usr/share/dotnet/sdk/8.0.303/Sdks/Microsoft.NET.SDK.WorkloadAutoImportPropsLocator/Sdk" did not exist.

Note that the restore and build itself, before, in separate steps, work without problems, however. The project itself is really very simple, here e.g. the whole .csproj of the ....Utils.API project:

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>net8.0</TargetFramework>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>
        <GeneratePackageOnBuild>True</GeneratePackageOnBuild>
        <Copyright>Copyright (c) *****. ALL RIGHTS RESERVED.</Copyright>
        <Company>*****</Company>
        <Authors>*****</Authors>
        <VersionPrefix>0.1.0</VersionPrefix>
        <VersionSuffix>alpha-$([System.DateTime]::Now.ToString("yyyyMMddHHmmss"))</VersionSuffix>
    </PropertyGroup>
</Project>

There arer other projects in the repo to be build/packaged, but AFAIK already this one fails.

Steps to Reproduce

My Azure DevOps pipeline fails at the state for "dotnet pack", where it should pack the .NET 8.0 project written in C#. Pipeline uses the Docker container mcr.microsoft.com/dotnet/sdk:8.0.

Relevant pipeline snippet:

// ...
      - job: Publish_NuGet
        displayName: "Publish NuGet Packages"
        workspace:
         clean: all
        container:
          image: $[ variables.docker_dotnet ]
          options: "$(docker_options_generic)"
        variables:
          DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
        steps:
          # checkout etc. stuff done here

          - task: DotNetCoreCLI@2
            displayName: "Restore NuGet packages"
            inputs:
              command: restore
              projects: "${{ parameters.projects }}"
              feedsToUse: 'select'
              vstsFeed: ${{ parameters.feed }}
              verbosityRestore: Normal
              includeNuGetOrg: ${{ parameters.includeNuGetOrg }}
              noCache: true
              # required for now as internal feed runs into rate-limiting/overload otherwise
              restoreArguments: '--disable-parallel'
            env:
              HTTP_PROXY: "$(proxy_http)"
              HTTPS_PROXY: "$(proxy_https)"
              NO_PROXY: "$(proxy_http_no)"

          - task: DotNetCoreCLI@2
            displayName: "Build Backend"
            inputs:
              projects: "$(workingDirectory)/**/*.csproj"
              arguments: "--no-restore --configuration $(lf_app_build_configuration) -o $(outputDirectory)"
              workingDirectory: $(workingDirectory)

          - template: "/templates/tasks/get-version-with-githash.yml"
            parameters:
              version: $(Build.BuildNumber)

          - task: DotNetCoreCLI@2
            displayName: "dotnet pack"
            inputs:
              command: pack
              projects: "$(workingDirectory)/**/*.csproj"
              outputDir: $(Build.ArtifactStagingDirectory)/module-packed
              nobuild: false
              # .symbols.nupkg are not to be generated, as they will conflict when pushed to the same feed later, instead
              # symbols are manually published to a file share as you can see below
              includesymbols: false
              includesource: true
              versioningScheme: byEnvVar
              versionEnvVar: lf_app_version

Here is the output error/log of dotnet pack: https://gist.github.com/rklec/7171e9fb3a764b4219e06732adc7deca

Other Information

Container used:

8.0: Pulling from dotnet/sdk
Digest: sha256:7d0ba26469267b563120456557e38eccef9972cb6b9cfbbd47a50d1218fa7b30
Status: Image is up to date for mcr.microsoft.com/dotnet/sdk:8.0
mcr.microsoft.com/dotnet/sdk:8.0

Also, the build itself does work:

``` The package *********.*******.Utils.Unittests.0.1.0-alpha-20240731105346 is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important. Successfully created package '/__w/2006/a/BuiltBackend/*********.*******.Utils.Unittests.0.1.0-alpha-20240731105346.nupkg'. *********.*******.Utils.Email -> /__w/2006/a/BuiltBackend/*********.*******.Utils.Email.dll *********.*******.Utils.Email.Test -> /__w/2006/a/BuiltBackend/*********.*******.Utils.Email.Test.dll Build succeeded. 0 Warning(s) 0 Error(s) Time Elapsed 00:00:03.87 /usr/bin/dotnet build /__w/2006/Utils/src/*********.*******.Utils.Email/*********.*******.Utils.Email.csproj -dl:CentralLogger,"/__w/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.221.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll"*ForwardingLogger,"/__w/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.221.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll" --no-restore --configuration Release -o /__w/2006/a/BuiltBackend *********.*******.Utils.API -> /__w/2006/a/BuiltBackend/*********.*******.Utils.API.dll The package *********.*******.Utils.API.0.1.0-alpha-20240731105350 is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important. Successfully created package '/__w/2006/a/BuiltBackend/*********.*******.Utils.API.0.1.0-alpha-20240731105350.nupkg'. *********.*******.Utils.Email -> /__w/2006/a/BuiltBackend/*********.*******.Utils.Email.dll Build succeeded. 0 Warning(s) 0 Error(s) Time Elapsed 00:00:01.75 /usr/bin/dotnet build /__w/2006/Utils/src/*********.*******.Utils.*********.Test/*********.*******.Utils.*********.Test.csproj -dl:CentralLogger,"/__w/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.221.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll"*ForwardingLogger,"/__w/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.221.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll" --no-restore --configuration Release -o /__w/2006/a/BuiltBackend *********.*******.Utils.Unittests -> /__w/2006/a/BuiltBackend/*********.*******.Utils.Unittests.dll The package *********.*******.Utils.Unittests.0.1.0-alpha-20240731105353 is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important. Successfully created package '/__w/2006/a/BuiltBackend/*********.*******.Utils.Unittests.0.1.0-alpha-20240731105353.nupkg'. *********.*******.Utils.********* -> /__w/2006/a/BuiltBackend/*********.*******.Utils.*********.dll The package *********.*******.Utils.*********.0.1.0-alpha-20240731105353 is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important. Successfully created package '/__w/2006/a/BuiltBackend/*********.*******.Utils.*********.0.1.0-alpha-20240731105353.nupkg'. *********.*******.Utils.*********.Test -> /__w/2006/a/BuiltBackend/*********.*******.Utils.*********.Test.dll Build succeeded. 0 Warning(s) 0 Error(s) Time Elapsed 00:00:02.16 /usr/bin/dotnet build /__w/2006/Utils/src/*********.*******.Utils.*********/*********.*******.Utils.*********.csproj -dl:CentralLogger,"/__w/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.221.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll"*ForwardingLogger,"/__w/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.221.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll" --no-restore --configuration Release -o /__w/2006/a/BuiltBackend *********.*******.Utils.********* -> /__w/2006/a/BuiltBackend/*********.*******.Utils.*********.dll The package *********.*******.Utils.*********.0.1.0-alpha-20240731105355 is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important. Successfully created package '/__w/2006/a/BuiltBackend/*********.*******.Utils.*********.0.1.0-alpha-20240731105355.nupkg'. Build succeeded. 0 Warning(s) 0 Error(s) Time Elapsed 00:00:01.23 /usr/bin/dotnet build /__w/2006/Utils/src/*********.*******.Utils.Unittests/*********.*******.Utils.Unittests.csproj -dl:CentralLogger,"/__w/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.221.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll"*ForwardingLogger,"/__w/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.221.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll" --no-restore --configuration Release -o /__w/2006/a/BuiltBackend *********.*******.Utils.Unittests -> /__w/2006/a/BuiltBackend/*********.*******.Utils.Unittests.dll The package *********.*******.Utils.Unittests.0.1.0-alpha-20240731105356 is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important. Successfully created package '/__w/2006/a/BuiltBackend/*********.*******.Utils.Unittests.0.1.0-alpha-20240731105356.nupkg'. Build succeeded. 0 Warning(s) 0 Error(s) Time Elapsed 00:00:01.29 Info: Azure Pipelines hosted agents have been updated and now contain .Net 5.x SDK/Runtime along with the older .Net Core version which are currently lts. Unless you have locked down a SDK version for your project(s), 5.x SDK might be picked up which might have breaking behavior as compared to previous versions. You can learn more about the breaking changes here: https://docs.microsoft.com/en-us/dotnet/core/tools/ and https://docs.microsoft.com/en-us/dotnet/core/compatibility/ . To learn about more such changes and troubleshoot, refer here: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/build/dotnet-core-cli?view=azure-devops#troubleshooting Finishing: Build Backend ```

Output of docker version

Only have this in the log:

/usr/bin/docker version --format '{{.Server.APIVersion}}'
'1.41'
Docker daemon API version: '1.41'
/usr/bin/docker version --format '{{.Client.APIVersion}}'
'1.41'
Docker client API version: '1.41'

Output of docker info

/usr/bin/docker info -f "{{range .Plugins.Network}}{{println .}}{{end}}"
bridge
host
ipvlan
macvlan
null
overlay
/usr/bin/docker pull mcr.microsoft.com/dotnet/sdk:8.0
8.0: Pulling from dotnet/sdk
Digest: sha256:7d0ba26469267b563120456557e38eccef9972cb6b9cfbbd47a50d1218fa7b30
Status: Image is up to date for mcr.microsoft.com/dotnet/sdk:8.0
mcr.microsoft.com/dotnet/sdk:8.0

Container runs on Ubuntu Linux

lbussell commented 3 months ago

As a sanity check, I validated that a basic scenario involving dotnet pack works using a Dockerfile:

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS sdk
ARG APP_NAME="MyLibrary"

# Boilerplate to create a sample project that we can pack
FROM sdk AS template
RUN dotnet new classlib --name $APP_NAME -f netstandard2.0 --no-restore -o /src

# Copy project files and restore
FROM sdk AS restore
WORKDIR /src/
COPY --from=template /src/*.csproj .
RUN dotnet restore

# Copy sources and build + pack
FROM restore AS build
COPY --from=template /src/. .
RUN dotnet build -c Release $APP_NAME.csproj
RUN dotnet pack --no-build -o /packages

This may be an issue with your project configuration, Azure DevOps, or MSBuild/the .NET SDK. I don't think it's an issue with the .NET container images, given that the basic scenario above works.

To isolate the issue, you could try to build your packages directly in the Docker container on your local machine (interactively or with a Dockerfile like above). You could also try and run the dotnet CLI directly using script tasks in your pipeline, instead of using the Azure DevOps tasks - I'm not sure what those are doing on the backend that could be messing with the .NET installation.

rklec commented 3 months ago

Ah yeah sorry it was the build placed it into the wrong outputDirectory, sorry.