nimbleways / dotnet-subset

A .NET tool that copies a subset of files from a repository to a directory. The tool is mainly used in Dockerfiles to optimize the docker build caching for "dotnet restore" instructions.
MIT License
88 stars 4 forks source link

Unhandled exception: Microsoft.Build.Exceptions.InvalidProjectFileException: The expression "[Microsoft.Build.Utilities.ToolLocationHelper]::GetPlatformSDKLocation('', '')" cannot be evaluated #9

Open othmane-kinane-nw opened 1 year ago

othmane-kinane-nw commented 1 year ago

solved by downgrade to --version 0.3.1, seems something wrong with 0.3.2


#22 [linux/arm64 prepare-restore-files 2/5] RUN dotnet tool install --global --no-cache dotnet-subset --version 0.3.2
#22 53.13 You can invoke the tool using the following command: dotnet-subset
#22 53.13 Tool 'dotnet-subset' (version '0.3.2') was successfully installed.
#22 DONE 53.4s

#34 [linux/arm64 prepare-restore-files 3/5] WORKDIR /src
#34 DONE 0.0s

#35 [linux/arm64 prepare-restore-files 4/5] COPY . .
#35 DONE 0.3s

#36 [linux/arm64 prepare-restore-files 5/5] RUN dotnet subset restore "Services/Luna/Ruby.Services.Luna.Api/Ruby.Services.Luna.Api.csproj" --root-directory /src --output /src/restore_subset/
#36 15.69 dotnet-subset 0.3.2 (.NET Runtime 7.0.5)
#36 36.83 Unhandled exception: Microsoft.Build.Exceptions.InvalidProjectFileException: The expression "[Microsoft.Build.Utilities.ToolLocationHelper]::GetPlatformSDKLocation('', '')" cannot be evaluated. Object reference not set to an instance of an object.  /usr/share/dotnet/sdk/7.0.203/Microsoft.Common.CurrentVersion.targets
#36 36.91    at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject(String errorSubCategoryResourceName, IElementLocation elementLocation, String resourceName, Object[] args)
#36 36.91    at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject[T1,T2](IElementLocation elementLocation, String resourceName, T1 arg0, T2 arg1)
#36 36.91    at Microsoft.Build.Evaluation.Expander`2.Function`1.Execute(Object objectInstance, IPropertyProvider`1 properties, ExpanderOptions options, IElementLocation elementLocation)
#36 36.91    at Microsoft.Build.Evaluation.Expander`2.PropertyExpander`1.ExpandPropertyBody(String propertyBody, Object propertyValue, IPropertyProvider`1 properties, ExpanderOptions options, IElementLocation elementLocation, UsedUninitializedProperties usedUninitializedProperties, IFileSystem fileSystem)
#36 36.91    at Microsoft.Build.Evaluation.Expander`2.PropertyExpander`1.ExpandPropertiesLeaveTypedAndEscaped(String expression, IPropertyProvider`1 properties, ExpanderOptions options, IElementLocation elementLocation, UsedUninitializedProperties usedUninitializedProperties, IFileSystem fileSystem, LoggingContext loggingContext)
#36 36.91    at Microsoft.Build.Evaluation.Expander`2.PropertyExpander`1.ExpandPropertiesLeaveEscaped(String expression, IPropertyProvider`1 properties, ExpanderOptions options, IElementLocation elementLocation, UsedUninitializedProperties usedUninitializedProperties, IFileSystem fileSystem, LoggingContext loggingContext)
#36 36.91    at Microsoft.Build.Evaluation.Expander`2.ExpandIntoStringLeaveEscaped(String expression, ExpanderOptions options, IElementLocation elementLocation, LoggingContext loggingContext)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.EvaluatePropertyElement(ProjectPropertyElement propertyElement)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.EvaluatePropertyGroupElement(ProjectPropertyGroupElement propertyGroupElement)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.Evaluate()
#36 36.91    at Microsoft.Build.Evaluation.Evaluator`4.Evaluate(IEvaluatorData`4 data, Project project, ProjectRootElement root, ProjectLoadSettings loadSettings, Int32 maxNodeCount, PropertyDictionary`1 environmentProperties, ILoggingService loggingService, IItemFactory`2 itemFactory, IToolsetProvider toolsetProvider, ProjectRootElementCacheBase projectRootElementCache, BuildEventContext buildEventContext, ISdkResolverService sdkResolverService, Int32 submissionId, EvaluationContext evaluationContext, Boolean interactive)
#36 36.91    at Microsoft.Build.Evaluation.Project.ProjectImpl.Reevaluate(ILoggingService loggingServiceForEvaluation, ProjectLoadSettings loadSettings, EvaluationContext evaluationContext)
#36 36.91    at Microsoft.Build.Evaluation.Project.ProjectImpl.ReevaluateIfNecessary(ILoggingService loggingServiceForEvaluation, ProjectLoadSettings loadSettings, EvaluationContext evaluationContext)
#36 36.91    at Microsoft.Build.Evaluation.Project.ProjectImpl.Initialize(IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectLoadSettings loadSettings, EvaluationContext evaluationContext)
#36 36.91    at Microsoft.Build.Evaluation.Project..ctor(String projectFile, IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectCollection projectCollection, ProjectLoadSettings loadSettings, EvaluationContext evaluationContext, IDirectoryCacheFactory directoryCacheFactory)
#36 36.91    at Microsoft.Build.Evaluation.ProjectCollection.LoadProject(String fileName, IDictionary`2 globalProperties, String toolsVersion)
#36 36.91    at Nimbleways.Tools.Subset.RestoreSubset.VisitAllProjects(ProjectCollection projectCollection, String rootFolder, String projectPath, Dictionary`2 projects) in /home/runner/work/dotnet-subset/dotnet-subset/src/dotnet-subset/RestoreSubset.cs:line 108
#36 36.91    at Nimbleways.Tools.Subset.RestoreSubset.VisitAllProjects(ProjectCollection projectCollection, String rootFolder, String projectPath, Dictionary`2 projects) in /home/runner/work/dotnet-subset/dotnet-subset/src/dotnet-subset/RestoreSubset.cs:line 114
#36 36.91    at Nimbleways.Tools.Subset.RestoreSubset.Execute(String projectOrSolution, String rootFolder, String destinationFolder) in /home/runner/work/dotnet-subset/dotnet-subset/src/dotnet-subset/RestoreSubset.cs:line 19
#36 36.91    at Program.<>c.<<Main>$>b__0_0(FileInfo projectOrSolution, DirectoryInfo rootDirectory, DirectoryInfo outputDirectory) in /home/runner/work/dotnet-subset/dotnet-subset/src/dotnet-subset/Program.cs:line 38
#36 36.91    at System.CommandLine.Handler.<>c__DisplayClass4_0`3.<SetHandler>b__0(InvocationContext context)
#36 36.91    at System.CommandLine.Invocation.AnonymousCommandHandler.Invoke(InvocationContext context)
#36 36.91    at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass17_0.<<UseParseErrorReporting>b__0>d.MoveNext()
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass12_0.<<UseHelp>b__0>d.MoveNext()
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<<UseVersionOption>b__0>d.MoveNext()
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass19_0.<<UseTypoCorrections>b__0>d.MoveNext()
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseSuggestDirective>b__18_0>d.MoveNext()
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass16_0.<<UseParseDirective>b__0>d.MoveNext()
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<RegisterWithDotnetSuggest>b__5_0>d.MoveNext()
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass8_0.<<UseExceptionHandler>b__0>d.MoveNext()
#36 ERROR: process "/bin/sh -c dotnet subset restore \"Services/Luna/Ruby.Services.Luna.Api/Ruby.Services.Luna.Api.csproj\" --root-directory /src --output /src/restore_subset/" did not complete successfully: exit code: 1
------
 > [linux/arm64 prepare-restore-files 5/5] RUN dotnet subset restore "Services/Luna/Ruby.Services.Luna.Api/Ruby.Services.Luna.Api.csproj" --root-directory /src --output /src/restore_subset/:
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass19_0.<<UseTypoCorrections>b__0>d.MoveNext()
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseSuggestDirective>b__18_0>d.MoveNext()
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass16_0.<<UseParseDirective>b__0>d.MoveNext()
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<RegisterWithDotnetSuggest>b__5_0>d.MoveNext()
#36 36.91 --- End of stack trace from previous location ---
#36 36.91    at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass8_0.<<UseExceptionHandler>b__0>d.MoveNext()
------
Dockerfile:13
--------------------
  11 |     WORKDIR /src
  12 |     COPY . .
  13 | >>> RUN dotnet subset restore "Services/Luna/Ruby.Services.Luna.Api/Ruby.Services.Luna.Api.csproj" --root-directory /src --output /src/restore_subset/
  14 |     
  15 |     FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
--------------------
ERROR: failed to solve: process "/bin/sh -c dotnet subset restore \"Services/Luna/Ruby.Services.Luna.Api/Ruby.Services.Luna.Api.csproj\" --root-directory /src --output /src/restore_subset/" did not complete successfully: exit code: 1
Error: buildx failed with: ERROR: failed to solve: process "/bin/sh -c dotnet subset restore \"Services/Luna/Ruby.Services.Luna.Api/Ruby.Services.Luna.Api.csproj\" --root-directory /src --output /src/restore_subset/" did not complete successfully: exit code: 1

Originally posted by @dmitriy-pisarevskiy in https://github.com/nimbleways/dotnet-subset/issues/6#issuecomment-1534159509

othmane-kinane-nw commented 1 year ago

@dmitriy-pisarevskiy can you please provide reproduction steps ? Ideally with a repository.

dmitriy-pisarevskiy commented 1 year ago

@othmane-kinane-nw hey, seems I solved it yesterday. working example net 7 Dockerfile using buildx and arm64 platform.

As I understood need to add amd64 layer for restore packages like mcr.microsoft.com/dotnet/sdk:7.0-bullseye-slim-amd64 in this case subset with version 0.3.2 is working fine, at least my builds all green.

I use next command to compile project:

docker buildx build --platform linux/arm64 --file src/Web/WebStatus/Dockerfile --build-arg FW_GITHUB_TOKEN=usertoken --build-arg FW_GITHUB_USER=username src/

FROM mcr.microsoft.com/dotnet/aspnet:7.0 AS base
WORKDIR /app
EXPOSE 80

FROM mcr.microsoft.com/dotnet/sdk:7.0-bullseye-slim-amd64 AS prepare-restore-files
ENV PATH="${PATH}:/root/.dotnet/tools"
RUN dotnet tool install --global --no-cache dotnet-subset --version 0.3.2
WORKDIR /src
COPY . .
RUN dotnet subset restore "Web/WebStatus/WebStatus.csproj" --root-directory /src --output /src/restore_subset/

FROM mcr.microsoft.com/dotnet/sdk:7.0-bullseye-slim-amd64 AS build
WORKDIR /src

COPY "Ruby.sln" "Ruby.sln"

COPY --from=prepare-restore-files /src/restore_subset .

COPY "docker-compose.dcproj" "docker-compose.dcproj"

COPY "NuGet.config" "NuGet.config"

ARG TARGETARCH
ARG TARGETOS

RUN arch=$TARGETARCH \
    && if [ "$arch" = "amd64" ]; then arch="x64"; fi \
    && echo $TARGETOS-$arch > /tmp/rid

ARG FW_GITHUB_TOKEN
ARG FW_GITHUB_USER

ENV GITHUB_PACKAGE_TOKEN=${FW_GITHUB_TOKEN}
ENV GITHUB_PACKAGE_USER_NAME=${FW_GITHUB_USER}

RUN dotnet restore "Web/WebStatus/WebStatus.csproj" -r $(cat /tmp/rid)

COPY . .

WORKDIR "/src/Web/WebStatus" 
RUN dotnet publish "WebStatus.csproj" --no-restore -c Release -o /app -r $(cat /tmp/rid) --self-contained false

FROM base AS final
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["dotnet", "WebStatus.dll"]
dmitriy-pisarevskiy commented 1 year ago

This appears to be a bug with NuGet running with QEMU (Quick Emulator). QEMU is being used here by Docker because you're running Arm64 on an x64 host machine.

In the meantime, let's talk about a workaround. You'll want to avoid emulation by not using a multi-arch tag for the sdk. To do this, you can use the 7.0-bullseye-slim-amd64 tag. You should only need to use that for the sdk stage of your Dockerfile. The final stage can still use the multi-arch tag (7.0).

But because you're explicitly using the amd64 version of the sdk image, you'll need to explicitly set the RID in the restore and publish commands. And because the RID is set you'll also need to disable the self-contained option if you don't want a self-contained deployment.

And there's one little gotcha with the architecture. In order to make it a truly multi-arch Dockerfile, you'd also need to account for when you specify --platform linux/amd64. In that case, the RID needs to use x64 not amd64, so there's some logic in the Dockerfile below which shows how to fix that up as well.

Note that in the Dockerfile below I make use of the TARGETARCH and TARGETOS arguments. You don't need to explicitly pass these arguments with the --build-arg option. They are automatically derived by using the --platform option. So you shouldn't need to change how you run the build command.