search

NuGet / Home

Repo for NuGet Client issues
Other
1.5k stars 252 forks source link

Clarify the inconsistency with restore and ignoring missing imports in dotnet/msbuild/nuget.exe restore #9035

Open nkolev92 opened 4 years ago

nkolev92 commented 4 years ago

Set-up a project with a missing import.

Try restoring:

dotnet restore

dotnet restore F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj                                                                                                                                       Nothing to do. None of the projects specified contain packages to restore.

msbuild restore

F:\VSExtensibility [develop ≡]> msbuild /t:restore F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj
Microsoft (R) Build Engine version 16.5.0-preview-19617-05+7bcb2f961 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 1/8/2020 4:50:52 PM.
Project "F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj" on node 1 (Restore target(s)).
Restore:
  Nothing to do. None of the projects specified contain packages to restore.
Done Building Project "F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj" (Restore target(s)).

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.52
F:\VSExtensibility [develop ≡]>

Define a restore2 target with the exact same content as the restore target. `

F:\VSExtensibility [develop ≡]> Invoke-NuGetCustom /t:restore2 F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj                                                                                                                     msbuild /p:NuGetRestoreTargets=F:\NuGet.Client\src\NuGet.Core\NuGet.Build.Tasks\NuGet.targets /p:RestoreTaskAssemblyFile=F:\NuGet.Client\artifacts\NuGet.Build.Tasks\16.0\bin\Debug\net472\NuGet.Build.Tasks.dll /p:NuGetBuildTasksPackTargets=F:\NuGet.Client\src\NuGet.Core\NuGet.Build.Tasks.Pack\NuGet.Build.Tasks.Pack.targets /p:ImportNuGetBuildTasksPackTargetsFromSdk=true /p:NuGetPackTaskAssemblyFile=F:\NuGet.Client\artifacts\NuGet.Build.Tasks.Pack\16.0\bin\Debug\net472\NuGet.Build.Tasks.Pack.dll /t:restore2 F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj
Microsoft (R) Build Engine version 16.5.0-preview-19617-05+7bcb2f961 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 1/8/2020 4:51:22 PM.
Project "F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj" on node 1 (restore2 target(s)).
F:\VSExtensibility\tools\build\targets\VSExtensibility.Common.Xplat.props(18,3): error MSB4019: The imported project "F:\VSExtensibility\packages\MicroBuild.VisualStudio.2.0.54\build\MicroBuild.VisualStudio.props" was not found. Confirm that the expression in the Import declaration "F:\VSExtensibility\packages\Micr
oBuild.VisualStudio.2.0.54\build\MicroBuild.VisualStudio.props" is correct, and that the file exists on disk. [F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj]
Done Building Project "F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj" (restore2 target(s)) -- FAILED.

Build FAILED.

"F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj" (restore2 target) (1) ->
  F:\VSExtensibility\tools\build\targets\VSExtensibility.Common.Xplat.props(18,3): error MSB4019: The imported project "F:\VSExtensibility\packages\MicroBuild.VisualStudio.2.0.54\build\MicroBuild.VisualStudio.props" was not found. Confirm that the expression in the Import declaration "F:\VSExtensibility\packages\Mi
croBuild.VisualStudio.2.0.54\build\MicroBuild.VisualStudio.props" is correct, and that the file exists on disk. [F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj]

    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:00.05
F:\VSExtensibility [develop ≡]>

nuget.exe restore

F:\VSExtensibility [develop ≡]> E:\NuGetExe\nuget-5.3.0.exe restore F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj -verbosity detailed
NuGet Version: 5.3.0.6260
MSBuild P2P timeout [ms]: 120000
MSBuild auto-detection: using msbuild version '16.5.0.61705' from 'C:\Program Files (x86)\Microsoft Visual Studio\2019\IntPreview\MSBuild\Current\Bin'. Use option -MSBuildVersion to force nuget to use a specific version of MSBuild.
C:\Program Files (x86)\Microsoft Visual Studio\2019\IntPreview\MSBuild\Current\Bin\msbuild.exe "C:\Users\nikolev.REDMOND\AppData\Local\Temp\NuGetScratch\04eijyto.iwq.nugetinputs.targets" /t:GenerateRestoreGraphFile /nologo /nr:false /v:q /p:NuGetRestoreTargets="C:\Users\nikolev.REDMOND\AppData\Local\Temp\NuGetScratch\x5btu2ag.fqf.nugetrestore.targets" /p:RestoreUseCustomAfterTargets="True" /p:RestoreTaskAssemblyFile="E:\NuGetExe\nuget-5.3.0.exe"
F:\VSExtensibility\tools\build\targets\VSExtensibility.Common.Xplat.props(18,3): error MSB4019: The imported project "F:\VSExtensibility\packages\MicroBuild.VisualStudio.2.0.54\build\MicroBuild.VisualStudio.props" was not found. Confirm that the expression in the Import declaration "F:\VSExtensibility\packages\MicroBuild.VisualStudio.2.0.54\build\MicroBuild.VisualStudio.props" is correct, and that the file exists on disk. [F:\VSExtensibility\src\product\vssdk\msbuild\Microsoft.VisualStudio.Sdk.BuildTasks.16.0\Microsoft.VisualStudio.Sdk.BuildTasks.16.0.csproj]

NuGet.CommandLine.ExitCodeException: Exception of type 'NuGet.CommandLine.ExitCodeException' was thrown.
   at NuGet.CommandLine.MsBuildUtility.<GetProjectReferencesAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.CommandLine.RestoreCommand.<GetDependencyGraphSpecAsync>d__68.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.CommandLine.RestoreCommand.<DetermineInputsFromMSBuildAsync>d__63.MoveNext()
WARNING: Error reading msbuild project information, ensure that your input solution or project file is valid. NETCore and UAP projects will be skipped, only packages.config files will be restored.
Restoring NuGet package MicroBuild.2.0.54.
Restoring NuGet package VS.ExternalAPIs.MSBuild.15.8.0-preview-000078.
Restoring NuGet package Microsoft.VisualStudioEng.MicroBuild.Core.0.4.1.
Restoring NuGet package Microsoft.VisualStudio.Setup.Common.2.5.1109-develop-ga454f805.
Restoring NuGet package Newtonsoft.Json.9.0.1.
Restoring NuGet package MicroBuild.VisualStudio.2.0.54.
Restoring NuGet package Nerdbank.GitVersioning.2.1.17.
Adding package 'MicroBuild.2.0.54' to folder 'F:\VSExtensibility\packages'
Adding package 'Microsoft.VisualStudio.Setup.Common.2.5.1109-develop-ga454f805' to folder 'F:\VSExtensibility\packages'
Adding package 'VS.ExternalAPIs.MSBuild.15.8.0-preview-000078' to folder 'F:\VSExtensibility\packages'
Adding package 'Nerdbank.GitVersioning.2.1.17' to folder 'F:\VSExtensibility\packages'
Adding package 'MicroBuild.VisualStudio.2.0.54' to folder 'F:\VSExtensibility\packages'
Adding package 'Newtonsoft.Json.9.0.1' to folder 'F:\VSExtensibility\packages'
Adding package 'Microsoft.VisualStudioEng.MicroBuild.Core.0.4.1' to folder 'F:\VSExtensibility\packages'
Added package 'MicroBuild.VisualStudio.2.0.54' to folder 'F:\VSExtensibility\packages'
Added package 'MicroBuild.VisualStudio.2.0.54' to folder 'F:\VSExtensibility\packages' from source 'C:\Users\nikolev.REDMOND\.nuget\packages\'
Added package 'MicroBuild.2.0.54' to folder 'F:\VSExtensibility\packages'
Added package 'MicroBuild.2.0.54' to folder 'F:\VSExtensibility\packages' from source 'C:\Users\nikolev.REDMOND\.nuget\packages\'
PackageSignatureVerificationLog: PackageIdentity: Nerdbank.GitVersioning.2.1.17 Source: C:\Users\nikolev.REDMOND\.nuget\packages\ PackageSignatureValidity: True
PackageSignatureVerificationLog: PackageIdentity: Microsoft.VisualStudio.Setup.Common.2.5.1109-develop-ga454f805 Source: C:\Users\nikolev.REDMOND\.nuget\packages\ PackageSignatureValidity: True
PackageSignatureVerificationLog: PackageIdentity: VS.ExternalAPIs.MSBuild.15.8.0-preview-000078 Source: C:\Users\nikolev.REDMOND\.nuget\packages\ PackageSignatureValidity: True
PackageSignatureVerificationLog: PackageIdentity: Microsoft.VisualStudioEng.MicroBuild.Core.0.4.1 Source: C:\Users\nikolev.REDMOND\.nuget\packages\ PackageSignatureValidity: True
Added package 'Microsoft.VisualStudioEng.MicroBuild.Core.0.4.1' to folder 'F:\VSExtensibility\packages'
Added package 'Microsoft.VisualStudioEng.MicroBuild.Core.0.4.1' to folder 'F:\VSExtensibility\packages' from source 'C:\Users\nikolev.REDMOND\.nuget\packages\'
Added package 'Newtonsoft.Json.9.0.1' to folder 'F:\VSExtensibility\packages'
Added package 'Newtonsoft.Json.9.0.1' to folder 'F:\VSExtensibility\packages' from source 'C:\Users\nikolev.REDMOND\.nuget\packages\'
Added package 'Microsoft.VisualStudio.Setup.Common.2.5.1109-develop-ga454f805' to folder 'F:\VSExtensibility\packages'
Added package 'Microsoft.VisualStudio.Setup.Common.2.5.1109-develop-ga454f805' to folder 'F:\VSExtensibility\packages' from source 'C:\Users\nikolev.REDMOND\.nuget\packages\'
Added package 'VS.ExternalAPIs.MSBuild.15.8.0-preview-000078' to folder 'F:\VSExtensibility\packages'
Added package 'VS.ExternalAPIs.MSBuild.15.8.0-preview-000078' to folder 'F:\VSExtensibility\packages' from source 'C:\Users\nikolev.REDMOND\.nuget\packages\'
Added package 'Nerdbank.GitVersioning.2.1.17' to folder 'F:\VSExtensibility\packages'
Added package 'Nerdbank.GitVersioning.2.1.17' to folder 'F:\VSExtensibility\packages' from source 'C:\Users\nikolev.REDMOND\.nuget\packages\'

NuGet Config files used:
    F:\VSExtensibility\NuGet.Config
    C:\Users\nikolev.REDMOND\AppData\Roaming\NuGet\NuGet.Config
    C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config
    C:\Program Files (x86)\NuGet\Config\Xamarin.Offline.config

Feeds used:
    C:\Users\nikolev.REDMOND\.nuget\packages\
    https://api.nuget.org/v3/index.json
    https://devdiv.pkgs.visualstudio.com/_packaging/VSExtensibility/nuget/v3/index.json
    https://devdiv.pkgs.visualstudio.com/DefaultCollection/_packaging/VS/nuget/v3/index.json
    https://devdiv.pkgs.visualstudio.com/_packaging/Setup/nuget/v3/index.json
    https://devdiv.pkgs.visualstudio.com/_packaging/vssdk/nuget/v3/index.json
    https://devdiv.pkgs.visualstudio.com/_packaging/vs-impl/nuget/v3/index.json
    https://devdiv.pkgs.visualstudio.com/_packaging/VSIDE-RealSigned-Release/nuget/v3/index.json

Installed:
    7 package(s) to packages.config projects
F:\VSExtensibility [develop ≡]>

This is inconsistent because of https://github.com/microsoft/msbuild/blob/8d37fbb97003e96516a2054e51dd5fabfd720ed6/src/MSBuild/XMake.cs#L1356.

The restore target is special cased.

Ideally all of the NuGet tooling is consistent.

The argument for ignoring missing imports is summarized in https://github.com/microsoft/msbuild/pull/3278. 

Some shipped SDKs have imports that are property based but are missing conditions. This makes it impossible to retrofit them with a NuGet package since NuGet restore won't succeed because an import is missing.

On the flip side, what if the imports in question might actually be genuinely missing cause of a setup issue. They might need to bring in a PackageReference themselves.

Regardless of what the decision is, all NuGet tooling should be consistent.

Related https://github.com/NuGet/Home/issues/7576

fyi @jeffkl @zivkan

Thoughts @NuGet/nuget-client?

zivkan commented 4 years ago

I think we investigated this weeks ago. What we found at the time:

https://github.com/microsoft/msbuild/blob/b111470ae61eba02c6102374c2b7d62aebe45f5b/src/MSBuild/XMake.cs#L1202

Which is used here:

https://github.com/microsoft/msbuild/blob/b111470ae61eba02c6102374c2b7d62aebe45f5b/src/MSBuild/XMake.cs#L1223-L1231

which calls a special ExecuteRestore method that sets additional flags:

https://github.com/microsoft/msbuild/blob/b111470ae61eba02c6102374c2b7d62aebe45f5b/src/MSBuild/XMake.cs#L1438

@nkolev92 should we close this, or is there something we want to act on?

nkolev92 commented 4 years ago

I think the actionable is the one from: https://github.com/NuGet/Home/issues/7576, where NuGet.exe behaves differelty from dotnet/msbuild restores.

So either making it consistent or documenting it.