The SDK 'Microsoft.NET.SDK.WorkloadAutoImportPropsLocator' specified could not be found

[grzesiek-galezowski]

O have an open source project (https://github.com/grzesiek-galezowski/nscan) that uses msbuild to load and evaluate a bunch of csproj files. I recently started using MSBuildLocator and I am running into an error when running my tests with .net 7 sdk installed.

When I enter the nscan\src\NScanSpecification.E2E and invoke dotnet test, I get an error when trying to create new instance of Microsoft.Build.Evaluation.Project. The error says:

Microsoft.Build.Exceptions.InvalidProjectFileException: The SDK 'Microsoft.NET.SDK.WorkloadAutoImportPropsLocator' specified could not be found.  C:\Program Files\dotnet\sdk\7.0.201\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.ImportWorkloads.props
   at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject(String errorSubCategoryResourceName, IElementLocation elementLocation, String resourceName, Object[] args)
   at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImportsFromUnescapedImportExpressionConditioned(String directoryOfImportingFile, ProjectImportElement importElement, List`1& projects, SdkResult& sdkResult, Boolean throwOnFileNotExistsError)
   at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImports(String directoryOfImportingFile, ProjectImportElement importElement, SdkResult& sdkResult)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.Evaluate()
   at Microsoft.Build.Evaluation.Project.ProjectImpl.Reevaluate(ILoggingService loggingServiceForEvaluation, ProjectLoadSettings loadSettings, EvaluationContext evaluationContext)
   at Microsoft.Build.Evaluation.Project.ProjectImpl.ReevaluateIfNecessary(ILoggingService loggingServiceForEvaluation, ProjectLoadSettings loadSettings, EvaluationContext evaluationContext)
   at Microsoft.Build.Evaluation.Project.ProjectImpl.ReevaluateIfNecessary(ILoggingService loggingServiceForEvaluation, EvaluationContext evaluationContext)
   at Microsoft.Build.Evaluation.Project.ProjectImpl.ReevaluateIfNecessary(EvaluationContext evaluationContext)
   at Microsoft.Build.Evaluation.Project.ProjectImpl.Initialize(IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectLoadSettings loadSettings, EvaluationContext evaluationContext)
   at Microsoft.Build.Evaluation.Project..ctor(ProjectRootElement xml, IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectCollection projectCollection, ProjectLoadSettings loadSettings, EvaluationContext evaluationContext, IDirectoryCacheFactory directoryCacheFactory)
   at Microsoft.Build.Evaluation.Project..ctor(ProjectRootElement xml, IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectCollection projectCollection, ProjectLoadSettings loadSettings)
   at Microsoft.Build.Evaluation.Project..ctor(ProjectRootElement xml, IDictionary`2 globalProperties, String toolsVersion, ProjectCollection projectCollection, ProjectLoadSettings loadSettings)
   at Microsoft.Build.Evaluation.Project..ctor(ProjectRootElement xml, IDictionary`2 globalProperties, String toolsVersion, ProjectCollection projectCollection)
   at Microsoft.Build.Evaluation.Project..ctor(ProjectRootElement xml, IDictionary`2 globalProperties, String toolsVersion)
   at Microsoft.Build.Evaluation.Project..ctor(ProjectRootElement xml)

This is when trying to load a project created for the test. Before using MsBuildLocator. My SDKs are:

Version      
-------------
3.1.426      
5.0.103      
5.0.201      
5.0.203      
5.0.214      
5.0.302      
5.0.408      
6.0.309      
6.0.402      
6.0.406      
7.0.201      

Name                             Version      
----------------------------------------------
Microsoft.NETCore.App             2.1.26      
Microsoft.NETCore.App             2.1.30      
Microsoft.AspNetCore.App          3.1.13      
Microsoft.NETCore.App             3.1.13      
Microsoft.WindowsDesktop.App      3.1.13      
Microsoft.AspNetCore.App          3.1.17      
Microsoft.NETCore.App             3.1.17      
Microsoft.WindowsDesktop.App      3.1.17      
Microsoft.AspNetCore.App          3.1.18      
Microsoft.NETCore.App             3.1.18      
Microsoft.WindowsDesktop.App      3.1.18      
Microsoft.AspNetCore.App          3.1.32      
Microsoft.NETCore.App             3.1.32      
Microsoft.WindowsDesktop.App      3.1.32      
Microsoft.AspNetCore.App          5.0.4       
Microsoft.NETCore.App             5.0.4       
Microsoft.WindowsDesktop.App      5.0.4       
Microsoft.NETCore.App             5.0.8       
Microsoft.AspNetCore.App          5.0.9       
Microsoft.NETCore.App             5.0.9       
Microsoft.WindowsDesktop.App      5.0.9       
Microsoft.AspNetCore.App          5.0.12      
Microsoft.NETCore.App             5.0.12      
Microsoft.WindowsDesktop.App      5.0.12      
Microsoft.NETCore.App             5.0.13      
Microsoft.WindowsDesktop.App      5.0.13      
Microsoft.AspNetCore.App          5.0.15      
Microsoft.NETCore.App             5.0.15      
Microsoft.WindowsDesktop.App      5.0.15      
Microsoft.AspNetCore.App          5.0.17      
Microsoft.NETCore.App             5.0.17      
Microsoft.WindowsDesktop.App      5.0.17      
Microsoft.NETCore.App             6.0.0       
Microsoft.WindowsDesktop.App      6.0.0       
Microsoft.NETCore.App             6.0.4       
Microsoft.WindowsDesktop.App      6.0.4       
Microsoft.AspNetCore.App          6.0.5       
Microsoft.NETCore.App             6.0.5       
Microsoft.WindowsDesktop.App      6.0.5       
Microsoft.AspNetCore.App          6.0.7       
Microsoft.NETCore.App             6.0.7       
Microsoft.WindowsDesktop.App      6.0.7       
Microsoft.AspNetCore.App          6.0.10      
Microsoft.NETCore.App             6.0.10      
Microsoft.WindowsDesktop.App      6.0.10      
Microsoft.NETCore.App             6.0.12      
Microsoft.AspNetCore.App          6.0.14      
Microsoft.NETCore.App             6.0.14      
Microsoft.WindowsDesktop.App      6.0.14      
Microsoft.AspNetCore.App          7.0.3       
Microsoft.NETCore.App             7.0.3       
Microsoft.WindowsDesktop.App      7.0.3       

and I am using this code in assembly initializer:

if (MSBuildLocator.CanRegister)
{
  MSBuildLocator.RegisterDefaults();
}

[Forgind]

How is MSBuild intended to find Microsoft.NET.SDK.WorkloadAutoImportPropsLocator? That is, which SDK resolver is supposed to find it, and can you verify whether it's running?

I also found this link that might help, since it seems like they're hitting a similar problem: https://learn.microsoft.com/answers/questions/478249/sdk-package-not-found-microsoft-net-sdk-workloadau

This doesn't look like an MSBuildLocator issue to me, but it might still be a bug in .NET.

[grzesiek-galezowski]

I tried the suggestion from that post and now my initializer code looks like this:

    Environment.SetEnvironmentVariable("MSBuildEnableWorkloadResolver", "false", EnvironmentVariableTarget.Process);
    if (MSBuildLocator.CanRegister)
    {
      MSBuildLocator.RegisterDefaults();
    }

but now I am getting an error that Microsoft.CSharp.targets is not found in the build output. I think that's not where it should be placed, so maybe there is some configuration missing? Let me know if this isn't MsBuild locator issue, then I'll stop bothering you. It's just that this started occuring after I started using MSBuild Locator and also it only occurs when running with dotnet.exe - when running from Visual Studio, everything works OK.

Microsoft.Build.Exceptions.InvalidProjectFileException: The imported project "C:\Users\HYPERBOOK\Documents\GitHub\nscan\src\NScanSpecification.E2E\bin\Debug\net7.0\Microsoft.CSharp.targets" was not found. Confirm that the expression in the Import declaration "C:\Users\HYPERBOOK\Documents\GitHub\nscan\src\NScanSpecification.E2E\bin\Debug\net7.0\Microsoft.CSharp.targets" is correct, and that the file exists on disk.  C:\Program Files\dotnet\sdk\7.0.202\Sdks\Microsoft.NET.Sdk\Sdk\Sdk.targets
   at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject(String errorSubCategoryResourceName, IElementLocation elementLocation, String resourceName, Object[] args)
   at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject[T1,T2](IElementLocation elementLocation, String resourceName, T1 arg0, T2 arg1)
   at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImportsFromUnescapedImportExpression(String directoryOfImportingFile, ProjectImportElement importElement, String unescapedExpression, Boolean throwOnFileNotExistsError, List`1& imports)
   at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImportsFromUnescapedImportExpressionConditioned(String directoryOfImportingFile, ProjectImportElement importElement, List`1& projects, SdkResult& sdkResult)

   at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImports(String directoryOfImportingFile, ProjectImportElement importElement, SdkResult& sdkResult)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.Evaluate()
   at Microsoft.Build.Evaluation.Evaluator`4.Evaluate(IEvaluatorData`4 data, Project project, ProjectRootElement root, ProjectLoadSettings loadSettings, Int32 maxNodeCount, PropertyDictionary`1 environmentProperties, ILoggingService loggingService, IItemFactory`2 itemFactory, IToolsetProvider toolsetProvider, ProjectRootElementCacheBase projectRootElementCache, BuildEventContext buildEventContext, ISdkResolverService sdkResolverService, Int32 submissionId, EvaluationContext evaluationContext, Boolean interactive)
   at Microsoft.Build.Evaluation.Project.ProjectImpl.Reevaluate(ILoggingService loggingServiceForEvaluation, ProjectLoadSettings loadSettings, EvaluationContext evaluationContext)
   at Microsoft.Build.Evaluation.Project.ProjectImpl.ReevaluateIfNecessary(ILoggingService loggingServiceForEvaluation, ProjectLoadSettings loadSettings, EvaluationContext evaluationContext)
   at Microsoft.Build.Evaluation.Project.ProjectImpl.ReevaluateIfNecessary(ILoggingService loggingServiceForEvaluation, EvaluationContext evaluationContext)
   at Microsoft.Build.Evaluation.Project.ProjectImpl.ReevaluateIfNecessary(EvaluationContext evaluationContext)
   at Microsoft.Build.Evaluation.Project.ProjectImpl.Initialize(IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectLoadSettings loadSettings, EvaluationContext evaluationContext)
   at Microsoft.Build.Evaluation.Project..ctor(ProjectRootElement xml, IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectCollection projectCollection, ProjectLoadSettings loadSettings, EvaluationContext evaluationContext, IDirectoryCacheFactory directoryCacheFactory)
   at Microsoft.Build.Evaluation.Project..ctor(ProjectRootElement xml, IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectCollection projectCollection, ProjectLoadSettings loadSettings)
   at Microsoft.Build.Evaluation.Project..ctor(ProjectRootElement xml, IDictionary`2 globalProperties, String toolsVersion, ProjectCollection projectCollection, ProjectLoadSettings loadSettings)
   at Microsoft.Build.Evaluation.Project..ctor(ProjectRootElement xml, IDictionary`2 globalProperties, String toolsVersion, ProjectCollection projectCollection)
   at Microsoft.Build.Evaluation.Project..ctor(ProjectRootElement xml, IDictionary`2 globalProperties, String toolsVersion)
   at Microsoft.Build.Evaluation.Project..ctor(ProjectRootElement xml)

[grzesiek-galezowski]

Hi, I was able to work around the issue by removing the MsBuildLocator, referencing Microsoft.CodeAnalysis.CSharp.Workspaces directly and replacing what I expected the MsBuildLocator to do with the following code: https://github.com/grzesiek-galezowski/nscan/blob/master/src/NScan.Lib/MsBuildPreconditions.cs

It uses the dotnet --list-sdks command to obtain the list of sdks, then constructs paths to all sdks, then sets the MSBUILD_EXE_PATH environment variable to an sdk compatible with the targeted dotnet version.

I expected this is what MsBuildLocator would do for me. Did I misunderstand the purpose of this tool?

[YuliiaKovalova]

Hi @grzesiek-galezowski,

The purpose of the tool is in locating and using a copy of MSBuild for build process. It looks like some configuration steps were missed. If you still need any assistance with the problem, feel free to reopen the ticket.

[grzesiek-galezowski]

@YuliiaKovalova What you wrote contradicts the README for this project which directly states that this nuget library is for C# applications that want to use the MSBuild C# API surface. So I take your answer as a more polite way of saying "we don't really care that you have problem". Fair enough. I found a workaround, so I can live with it.

All the best!

[Forgind]

"uses msbuild to load and evaluate a bunch of csproj files" is definitely a supported use of MSBuildLocator. I suspect that by "build process," YuliiaKovalova was including evaluating a project, as that's part of MSBuild even though it isn't what most people think of when they think of building.

I suspect she just closed this because it sounded like you were unblocked. Looking at what you did, it looks pretty similar to what MSBuildLocator is supposed to do (or rather, what it did before it switched to using hostfxr) except that MSBuildLocator just adds MSBuild assemblies to your assembly resolver, whereas you added MSBuild's location in the environment variable MSBUILD_EXE_PATH. (MSBuildLocator may have worked for you in VS because we do that, too—if you're on framework.)

Perhaps we should make that always run? What do you think?

[grzesiek-galezowski]

Hi, If this is the difference, then there's a chance it will work. If you have a test build, I can test it with my code 👍.

[Forgind]

I've been thinking about this, and I've convinced myself that setting MSBUILD_EXE_PATH works, but it's not the ideal answer. I think what's happening is that we're loading Microsoft.Build.dll, and it then tries to find things that are supposed to be next to it. Since it worked when you set MSBUILD_EXE_PATH, I assume that those files (including Microsoft.CSharp.targets) are indeed next to it, so you don't have a broken install. But Microsoft.Build.dll is looking in the wrong places for the files. First it looks at MSBUILD_EXE_PATH. MSBuildLocator doesn't set that on core, so that fails. Then it looks next to itself, but it thinks it's running in either dotnet.exe or msbuild.exe, and we know where those are and look at the right path relative to those—but in this case, we're wrong on both counts since we're in yourapp.exe instead. Then it just gives up and throws an error.

I would have to do a deeper analysis to be sure, but I currently believe that this is a legitimate bug, but it's a bug on MSBuild, not MSBuildLocator, and that MSBuild should be hardened against the case of "Microsoft.Build.dll was loaded into a random process."

I don't actually work on MSBuild or MSBuildLocator anymore, though, so I'll let YuliiaKovalova triage that 🙂

[rainersigwald]

@grzesiek-galezowski in your repo it looks like MSBuild assemblies are being placed next to your outputs because of the way you're referencing MSBuild. In this case MSBuildLocator does nothing. I suspect the right fix for you is to change your build process to avoid that copy (so you can use MSBuild from the right SDK instead of the specific old version you're referencing) and reinstate Locator.

@forgind I don't understand why you think there's a bug anywhere in MSBuild or Locator for this. Can you elaborate?

[Forgind]

The reference to MSBuild seems to be coming indirectly through Buildalyzer. That package only requires 17.0.1, so that's what it finds, but it would be better if it found the version in the SDK.

I think a lot of apps are supposed to have all their dependencies in their output folders—is that true? So should it just explicitly exclude MSBuild assemblies?

I'd previously assumed that there was only one version of MSBuild involved, and that we were finding it (when using Locator) and loading its assemblies into the current process (myapp.exe) but not providing any way to find associated files because the failure was in looking for Microsoft.CSharp.targets instead of in looking for Microsoft.Build.dll. Setting MSBUILDEXEPATH apparently resolved the problem, so Microsoft.CSharp.targets had to be present somewhere; it just wasn't found.

I then postulated that it wasn't found because of a bug in MSBuild somewhere. An alternate reason for it to not be found is that MSBuild had been copied somewhere without Microsoft.CSharp.targets, and it sounds like the latter is what happened in this case.

[rainersigwald]

The reference to MSBuild seems to be coming indirectly through Buildalyzer. That package only requires 17.0.1, so that's what it finds, but it would be better if it found the version in the SDK.

At compile time an older version is completely fine, it only sets a floor on the SDK version that would be compatible.

I think a lot of apps are supposed to have all their dependencies in their output folders—is that true? So should it just explicitly exclude MSBuild assemblies?

That is generally how .NET references work, yes, but it doesn't work for MSBuild because we don't want to use a fixed version of MSBuild, we want to use the right one for the runtime environment, which is what Locator does.

[grzesiek-galezowski]

So the root cause here is that another dependency brings in MSBuild assemblies into the output folder while they are not supposed to be there because MSBuildLocator should find the right ones on the system?

[rainersigwald]

That's my guess based on building the current version of your repo, but I didn't recreate the Locator stuff and the specific failure you observed.

[grzesiek-galezowski]

Thanks, I will try to take a look today and see if that's the case.

[grzesiek-galezowski]

Hi, I removed the dependency on Buildalyzer added MsBuildLocator and indeed the tests are passing using the VS Test Runner and Resharper Runner. They still don't work in NCrunch but that's between me and that tool. So I'm OK with closing this issue.