MSB0001: Internal MSBuild Error: Node does not have a provider.

[cdmihai]

We got a report of an intermittent msbuild failure (~once every 2 weeks) with the following message:

UNHANDLED EXCEPTIONS FROM PROCESS 4544: 
===================== 

Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Node 2 does not have a provider. 
at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Boolean needToValidateProject, String schemaFile, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, Boolean debugger, Boolean detailedSummary) 
at Microsoft.Build.CommandLine.MSBuildApp.Execute(String commandLine) 
at Microsoft.Build.CommandLine.MSBuildApp.Main() 
=================== 

Context details: 6 C# projects, built on VS2015. The solution is a VS2010 solution with target .net 4.0.

Cmdline (with replaced project names): Executing the command cmd.exe /C " "C:\Program Files (x86)\MSBuild\14.0\Bin\msbuild.exe" /t:Clean,Build /P:Configuration=Release;Platform=x86 /maxcpucount ProjectFolder\Project.sln " && exit %%ERRORLEVEL%% from H:\Jenkins\workspace\Dev_Build [Dev_Build] $ cmd.exe /C " "C:\Program Files (x86)\MSBuild\14.0\Bin\msbuild.exe" /t:Clean,Build /P:Configuration=Release;Platform=x86 /maxcpucount ProjectFolder\Project.sln " && exit %%ERRORLEVEL%%

[cdmihai]

I looked over the code to see how a node can get to have a null provider.

It seems possible to get into this state. One would have to:

• Shutdown a node (BuildManager.Shutdown())
• Send message to that node (BuildManager.PerformActions(), which at some point leads to NodeManager.SendData())

I also saw some potential concurrency concerns that might lead to this interleaving:

• Escaped node ints: When a node is created, an int token is created as a reference for that node. The node manager just seems to accept these tokens without validation, and I did not find code that invalidates an int node reference. In consequence, it could be possible to have an escaped int token representing a node that was shut down still circulating in the node manager and build manager.
• NodeManager._NodesShutdown is not volatile. Are the possible callers confined to the same thread? If not, stale cache values of this boolean could cause missed shutdown messages to node manager. This could lead to messages being passed to nodes after shutdown.
• BuildManager._ShuttingDown is not volatile. Same story as above. Accessed without a lock in BuildManager.Reset()
• There's use of non thread safe collections (_activeNodes field in BuildManager)

Next steps:

• see if we can write a unit test invoking the BuildManager with the above steps that exposes this interleaving

[cdmihai]

New dump with stack available in internal issue: https://devdiv.visualstudio.com/DevDiv/_workitems?id=229584&_a=edit

Summary of interesting things from the stack: InProcNode:

-       this    {Microsoft.Build.BackEnd.InProcNode}    Microsoft.Build.BackEnd.InProcNode
+       buildRequestEngine  {Microsoft.Build.BackEnd.BuildRequestEngine}    Microsoft.Build.BackEnd.IBuildRequestEngine {Microsoft.Build.BackEnd.BuildRequestEngine}
+       componentHost   {Microsoft.Build.Execution.BuildManager}    Microsoft.Build.BackEnd.IBuildComponentHost {Microsoft.Build.Execution.BuildManager}
+       configurationProjectsLoaded Count = 0   System.Collections.Generic.HashSet<Microsoft.Build.Shared.NGen<int>>
+       currentConfiguration    {Microsoft.Build.BackEnd.NodeConfiguration} Microsoft.Build.BackEnd.NodeConfiguration
+       engineExceptionEventHandler {Method = Inspecting the state of an object in the debuggee of type System.Delegate is not supported while minidump debugging.} Microsoft.Build.BackEnd.EngineExceptionDelegate
        loggingContext  null    Microsoft.Build.BackEnd.Logging.NodeLoggingContext
+       newConfigurationRequestEventHandler {Method = Inspecting the state of an object in the debuggee of type System.Delegate is not supported while minidump debugging.} Microsoft.Build.BackEnd.NewConfigurationRequestDelegate
+       nodeEndpoint    {Microsoft.Build.BackEnd.NodeEndpointInProc}    Microsoft.Build.BackEnd.INodeEndpoint {Microsoft.Build.BackEnd.NodeEndpointInProc}
+       packetReceivedEvent {System.Threading.AutoResetEvent}   System.Threading.AutoResetEvent
+       receivedPackets Count = 0   System.Collections.Generic.Queue<Microsoft.Build.BackEnd.INodePacket>
+       requestBlockedEventHandler  {Method = Inspecting the state of an object in the debuggee of type System.Delegate is not supported while minidump debugging.} Microsoft.Build.BackEnd.RequestBlockedDelegate
+       requestCompleteEventHandler {Method = Inspecting the state of an object in the debuggee of type System.Delegate is not supported while minidump debugging.} Microsoft.Build.BackEnd.RequestCompleteDelegate
        savedCurrentDirectory
+       savedEnvironment    Count = 76  System.Collections.Generic.IDictionary<string, string> {System.Collections.Generic.Dictionary<string, string>}
+       shutdownEvent   {System.Threading.AutoResetEvent}   System.Threading.AutoResetEvent
+       shutdownException   null    System.Exception
        shutdownReason  BuildCompleteReuse  Microsoft.Build.Execution.NodeEngineShutdownReason

NodeProviderInProc:

-       this    {Microsoft.Build.BackEnd.NodeProviderInProc}    Microsoft.Build.BackEnd.NodeProviderInProc
        AvailableNodes  1   int
        ProviderType    InProc  Microsoft.Build.BackEnd.NodeProviderType
        componentHost   null    Microsoft.Build.BackEnd.IBuildComponentHost
        disposed    true    bool
+       endpointConnectedEvent  null    System.Threading.AutoResetEvent
        exclusiveOperatingEnvironment   false   bool
        inProcNode  null    Microsoft.Build.BackEnd.INode
+       inProcNodeEndpoint  {Microsoft.Build.BackEnd.NodeEndpointInProc}    Microsoft.Build.BackEnd.INodeEndpoint {Microsoft.Build.BackEnd.NodeEndpointInProc}
        inProcNodeId    0   int
+       inProcNodeOwningOperatingEnvironment    null    System.Threading.Semaphore
+       inProcNodeThread    {System.Threading.Thread}   System.Threading.Thread
+       packetFactory   {Microsoft.Build.BackEnd.NodeManager}   Microsoft.Build.BackEnd.INodePacketFactory {Microsoft.Build.BackEnd.NodeManager}
+       Static members      

BuildManager:

-       componentHost   {Microsoft.Build.Execution.BuildManager}    Microsoft.Build.BackEnd.IBuildComponentHost {Microsoft.Build.Execution.BuildManager}
+       Microsoft.Build.BackEnd.IBuildComponentHost.BuildParameters {Microsoft.Build.Execution.BuildParameters} Microsoft.Build.Execution.BuildParameters
+       Microsoft.Build.BackEnd.IBuildComponentHost.LegacyThreadingData {Microsoft.Build.Execution.LegacyThreadingData} Microsoft.Build.Execution.LegacyThreadingData
+       Microsoft.Build.BackEnd.IBuildComponentHost.LoggingService  'this.componentHost.LoggingService' threw an exception of type 'System.NullReferenceException'  Microsoft.Build.BackEnd.Logging.ILoggingService {System.NullReferenceException}
        Microsoft.Build.BackEnd.IBuildComponentHost.Name    "Default"   string
        acquiredProjectRootElementCacheFromProjectInstance  false   bool
+       activeNodes Count = 0   System.Collections.Generic.HashSet<Microsoft.Build.Shared.NGen<int>>
        buildManagerState   Idle    Microsoft.Build.Execution.BuildManager.BuildManagerState
+       buildParameters {Microsoft.Build.Execution.BuildParameters} Microsoft.Build.Execution.BuildParameters
+       buildSubmissions    Count = 0   System.Collections.Generic.Dictionary<int, Microsoft.Build.Execution.BuildSubmission>
+       componentFactories  {Microsoft.Build.BackEnd.BuildComponentFactoryCollection}   Microsoft.Build.BackEnd.BuildComponentFactoryCollection
+       configCache {Microsoft.Build.BackEnd.ConfigCache}   Microsoft.Build.BackEnd.IConfigCache {Microsoft.Build.BackEnd.ConfigCache}
        disposed    true    bool
        hostName    "Default"   string
+       legacyThreadingData {Microsoft.Build.Execution.LegacyThreadingData} Microsoft.Build.Execution.LegacyThreadingData
+       loggingThreadExceptionEventHandler  {Method = Inspecting the state of an object in the debuggee of type System.Delegate is not supported while minidump debugging.} Microsoft.Build.BackEnd.Logging.LoggingExceptionDelegate
        nextBuildSubmissionId   1   int
        nextUnnamedProjectId    1   int
+       noActiveSubmissionsEvent    null    System.Threading.AutoResetEvent
+       noNodesActiveEvent  null    System.Threading.AutoResetEvent
        nodeConfiguration   null    Microsoft.Build.BackEnd.NodeConfiguration
+       nodeIdToKnownConfigurations Count = 0   System.Collections.Generic.Dictionary<Microsoft.Build.Shared.NGen<int>, System.Collections.Generic.HashSet<Microsoft.Build.Shared.NGen<int>>>
        nodeManager null    Microsoft.Build.BackEnd.INodeManager
        overallBuildSuccess true    bool
+       projectFinishedEventHandler {Method = Inspecting the state of an object in the debuggee of type System.Delegate is not supported while minidump debugging.} Microsoft.Build.Framework.ProjectFinishedEventHandler
+       projectStartedEventHandler  {Method = Inspecting the state of an object in the debuggee of type System.Delegate is not supported while minidump debugging.} Microsoft.Build.Framework.ProjectStartedEventHandler
+       projectStartedEvents    Count = 0   System.Collections.Generic.Dictionary<int, Microsoft.Build.Framework.BuildEventArgs>
+       resultsCache    {Microsoft.Build.BackEnd.ResultsCache}  Microsoft.Build.BackEnd.IResultsCache {Microsoft.Build.BackEnd.ResultsCache}
        scheduler   null    Microsoft.Build.BackEnd.IScheduler
        shuttingDown    false   bool
        syncLock    {object}    object
+       taskHostNodeManager {Microsoft.Build.BackEnd.TaskHostNodeManager}   Microsoft.Build.BackEnd.INodeManager {Microsoft.Build.BackEnd.TaskHostNodeManager}
+       threadException {"MSB0001: Internal MSBuild Error: Node 2 does not have a provider."}   System.Exception {Microsoft.Build.Shared.InternalErrorException}
+       unnamedProjectInstanceToNames   Count = 0   System.Collections.Generic.Dictionary<Microsoft.Build.Execution.ProjectInstance, string>
        workQueue   null    System.Threading.Tasks.Dataflow.ActionBlock<System.Action>
+       Static members      
+       configurationProjectsLoaded Count = 0   System.Collections.Generic.HashSet<Microsoft.Build.Shared.NGen<int>>
+       currentConfiguration    {Microsoft.Build.BackEnd.NodeConfiguration} Microsoft.Build.BackEnd.NodeConfiguration
+       engineExceptionEventHandler {Method = Inspecting the state of an object in the debuggee of type System.Delegate is not supported while minidump debugging.} Microsoft.Build.BackEnd.EngineExceptionDelegate
        loggingContext  null    Microsoft.Build.BackEnd.Logging.NodeLoggingContext
+       newConfigurationRequestEventHandler {Method = Inspecting the state of an object in the debuggee of type System.Delegate is not supported while minidump debugging.} Microsoft.Build.BackEnd.NewConfigurationRequestDelegate
+       nodeEndpoint    {Microsoft.Build.BackEnd.NodeEndpointInProc}    Microsoft.Build.BackEnd.INodeEndpoint {Microsoft.Build.BackEnd.NodeEndpointInProc}
+       packetReceivedEvent {System.Threading.AutoResetEvent}   System.Threading.AutoResetEvent
+       receivedPackets Count = 0   System.Collections.Generic.Queue<Microsoft.Build.BackEnd.INodePacket>
+       requestBlockedEventHandler  {Method = Inspecting the state of an object in the debuggee of type System.Delegate is not supported while minidump debugging.} Microsoft.Build.BackEnd.RequestBlockedDelegate
+       requestCompleteEventHandler {Method = Inspecting the state of an object in the debuggee of type System.Delegate is not supported while minidump debugging.} Microsoft.Build.BackEnd.RequestCompleteDelegate
        savedCurrentDirectory   "H:\\Jenkins\\workspace\\GUI_1.4.0.0_Dev_Build" string
+       savedEnvironment    Count = 76  System.Collections.Generic.IDictionary<string, string> {System.Collections.Generic.Dictionary<string, string>}
+       shutdownEvent   {System.Threading.AutoResetEvent}   System.Threading.AutoResetEvent
+       shutdownException   null    System.Exception
        shutdownReason  BuildCompleteReuse  Microsoft.Build.Execution.NodeEngineShutdownReason

BuildManager has shuttingDown = False, nodeManager = null, disposed = true, and buildManagerState = Idle suggests that BuildManager.Reset was called. This only gets call on EndBuild, which looks like it's called from the command line after the build finished.

[wilstoff]

We have been getting this same issue intermittently as well. Any updates? or if it has been fixed in a newer version?

 **********************************************************************

** Visual Studio 2017 Developer Command Prompt v15.6.4

** Copyright (c) 2017 Microsoft Corporation

**********************************************************************

MSBUILD : error MSB4166: Child node "4" exited prematurely. Shutting down. Diagnostic information may be found in files in the temporary files directory named MSBuild_*.failure.txt.

Build succeeded.

    0 Warning(s)

    1 Error(s)

Time Elapsed 00:00:01.15

MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.

Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Node 4 does not have a provider.

   at Microsoft.Build.Execution.BuildManager.EndBuild()

   at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Boolean needToValidateProject, String schemaFile, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, Boolean debugger, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler)

MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.

Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Node 4 does not have a provider.

   at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Boolean needToValidateProject, String schemaFile, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, Boolean debugger, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler)

   at Microsoft.Build.CommandLine.MSBuildApp.Execute(String commandLine)

Unhandled Exception: Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Node 4 does not have a provider.

   at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Boolean needToValidateProject, String schemaFile, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, Boolean debugger, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler)

   at Microsoft.Build.CommandLine.MSBuildApp.Execute(String commandLine)

   at Microsoft.Build.CommandLine.MSBuildApp.Main()

[cdmihai]

@wilstoff It hasn't been fixed. Do you have a repro you can zip up and attach here?

[wilstoff]

Nothing i can attach, all work related code. It's in a code base with about 95 solutions, that we build in parallel tiers based on dependencies. So it could be something where we are calling msbuild too many times from one box in parallel (even if for different solutions). It's not easily recreate-able and we have since lost any data from this particular build.

[vok1980]

Got the same problem:

UNHANDLED EXCEPTIONS FROM PROCESS 716:
=====================
1/18/2019 8:28:43 PM
Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Node 2 does not have a provider.
   at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Boolean needToValidateProject, String schemaFile, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler)
   at Microsoft.Build.CommandLine.MSBuildApp.Execute(String commandLine)
   at Microsoft.Build.CommandLine.MSBuildApp.Main()
===================

Microsoft Visual Studio Professional 2017 Version 15.9.4 VisualStudio.15.Release/15.9.4+28307.222

[vok1980]

Is there any workaround for it?

[cdmihai]

@vok1980 turning off parallel builds would probably fix it. If you have a consistent repro, feel free to share it.

[vok1980]

I had a problem with sequential builds on CI. Looks like msbuild does not finish process after build immediately and some collision with next build happens. And I found that option /nodeReuse:false solves the problem.

[rainersigwald]

msbuild does not finish process after build immediately

True, when /nodeReuse:true (which is the default).

some collision with next build happens

That shouldn't happen, and doesn't in most cases. If you can narrow down any repro steps, we'd love to know them so we can fix the bug.

[vok1980]

Unfortunately, the only thing I can tell you about my project that its rather big. Build time is about 1h40m with 8 cores & 16G mem. And it does not matter if builds are taking place on the same workspace or separate workspaces. Hope, it can help you.

[williambrode]

@vok1980 I'm seeing the issue too and haven't tried /nodeReuse:false yet. But how confident are you that this setting fixes it? I only ask if it was intermittent and hard to reproduce if you've just gotten lucky so far.

[vok1980]

@williambrode It was 100% reproducing in my case. Now its 0% reproducing for a long time in the same case.

[danmoseley]

If your build time is that long, /nodeReuse:false would not affect perf measurably. It's an optimization for very quick, probably up to date builds of one or two projects.

If it's in VS, I am not sure whether VS now has a setting to disable node reuse, it used to be accomplished by setting an env var MSBUILDDISABLENODEREUSE=1.

[vaboca]

Hi everyone, I got a consistent repro when enabling /graph on our root node project (with or without /nr:false)

MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.
Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Node 2 does not have a provider.
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInternalError(String message, Object[] args)
   at Microsoft.Build.BackEnd.NodeManager.SendData(Int32 node, INodePacket packet)
   at Microsoft.Build.Execution.BuildManager.PerformSchedulingActions(IEnumerable`1 responses)
   at Microsoft.Build.Execution.BuildManager.ProcessPacket(Int32 node, INodePacket packet)
   at Microsoft.Build.Execution.BuildManager.ProcessWorkQueue(Action action)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Build.Execution.BuildManager.EndBuild()
   at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Boolean needToValidateProject, String schemaFile, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, TextWriter targetsWriter, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler, Boolean interactive, Boolean isolateProjects, Boolean graphBuild, Boolean lowPriority, String[] inputResultsCaches, String outputResultsCache)

MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.
Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Node 2 does not have a provider.
   at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Boolean needToValidateProject, String schemaFile, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, TextWriter targetsWriter, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler, Boolean interactive, Boolean isolateProjects, Boolean graphBuild, Boolean lowPriority, String[] inputResultsCaches, String outputResultsCache)
   at Microsoft.Build.CommandLine.MSBuildApp.Execute(String commandLine)

Unhandled Exception: Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Node 2 does not have a provider.
   at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Boolean needToValidateProject, String schemaFile, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, TextWriter targetsWriter, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler, Boolean interactive, Boolean isolateProjects, Boolean graphBuild, Boolean lowPriority, String[] inputResultsCaches, String outputResultsCache)
   at Microsoft.Build.CommandLine.MSBuildApp.Execute(String commandLine)
   at Microsoft.Build.CommandLine.MSBuildApp.Main()

------

0:000> k
 # Child-SP          RetAddr               Call Site
00 00000040`4ddfb940 00007ff8`3c5b0105     KERNELBASE!RaiseException+0x69 [minkernel\kernelbase\xcpt.c @ 937] 
01 00000040`4ddfba20 00007ff8`3c5b8847     clr!RaiseTheExceptionInternalOnly+0x31f [f:\dd\ndp\clr\src\vm\excep.cpp @ 3065] 
02 00000040`4ddfbb40 00007fff`fce51aa9     clr!IL_Rethrow+0xb7 [f:\dd\ndp\clr\src\vm\jithelpers.cpp @ 5179] 
03 00000040`4ddfbc90 00007ff8`3c5aee12     MSBuild_ni!Microsoft.Build.CommandLine.MSBuildApp.Execute+0xd99 [/_/src/MSBuild/XMake.cs @ 808] 
04 00000040`4ddfbdd0 00007ff8`3c5aece8     clr!ExceptionTracker::CallHandler+0xee [f:\dd\ndp\clr\src\vm\exceptionhandling.cpp @ 3323] 
05 00000040`4ddfbe80 00007ff8`3c5aebee     clr!ExceptionTracker::CallCatchHandler+0x9c [f:\dd\ndp\clr\src\vm\exceptionhandling.cpp @ 547] 
06 00000040`4ddfbf20 00007ff8`4907138f     clr!ProcessCLRException+0x31c [f:\dd\ndp\clr\src\vm\exceptionhandling.cpp @ 1062] 
07 00000040`4ddfc000 00007ff8`48ff1d19     ntdll!RtlpExecuteHandlerForUnwind+0xf [minkernel\ntos\rtl\amd64\xcptmisc.asm @ 254] 
08 00000040`4ddfc030 00007ff8`3c5afac0     ntdll!RtlUnwindEx+0x339 [minkernel\ntos\rtl\amd64\exdsptch.c @ 1425] 
09 00000040`4ddfc750 00007ff8`3c5afa7e     clr!ClrUnwindEx+0x40 [f:\dd\ndp\clr\src\vm\exceptionhandling.cpp @ 4298] 
0a 00000040`4ddfcc70 00007ff8`4907130f     clr!ProcessCLRException+0x2e9 [f:\dd\ndp\clr\src\vm\exceptionhandling.cpp @ 1025] 
0b 00000040`4ddfcd50 00007ff8`4901b5e4     ntdll!RtlpExecuteHandlerForException+0xf [minkernel\ntos\rtl\amd64\xcptmisc.asm @ 132] 
0c 00000040`4ddfcd80 00007ff8`4901b335     ntdll!RtlDispatchException+0x244 [minkernel\ntos\rtl\amd64\exdsptch.c @ 595] 
0d 00000040`4ddfd490 00007ff8`46a6d759     ntdll!RtlRaiseException+0x185 [minkernel\ntos\rtl\amd64\raise.c @ 194] 
0e 00000040`4ddfe340 00007ff8`3c5b0105     KERNELBASE!RaiseException+0x69 [minkernel\kernelbase\xcpt.c @ 937] 
0f 00000040`4ddfe420 00007ff8`3c5b0c14     clr!RaiseTheExceptionInternalOnly+0x31f [f:\dd\ndp\clr\src\vm\excep.cpp @ 3065] 
10 00000040`4ddfe540 00007fff`fce52a60     clr!IL_Throw+0x114 [f:\dd\ndp\clr\src\vm\jithelpers.cpp @ 5148] 
11 00000040`4ddfe6f0 00007fff`fce513c1     MSBuild_ni!Microsoft.Build.CommandLine.MSBuildApp.BuildProject+0xd40 [/_/src/MSBuild/XMake.cs @ 1280] 
12 00000040`4ddfe8a0 00007fff`fce50256     MSBuild_ni!Microsoft.Build.CommandLine.MSBuildApp.Execute+0x6b1 [/_/src/MSBuild/XMake.cs @ 651] 
13 00000040`4ddfeb00 00007ff8`3c3f6913     MSBuild_ni!Microsoft.Build.CommandLine.MSBuildApp.Main+0x66 [/_/src/MSBuild/XMake.cs @ 221] 
14 00000040`4ddfeb40 00007ff8`3c3f6824     clr!CallDescrWorkerInternal+0x83 [f:\dd\ndp\clr\src\vm\amd64\CallDescrWorkerAMD64.asm @ 97] 
15 00000040`4ddfeb80 00007ff8`3c3f70d8     clr!CallDescrWorkerWithHandler+0x4e [f:\dd\ndp\clr\src\vm\callhelpers.cpp @ 89] 
16 00000040`4ddfebc0 00007ff8`3c50b557     clr!MethodDescCallSite::CallTargetWorker+0x102 [f:\dd\ndp\clr\src\vm\callhelpers.cpp @ 655] 
17 (Inline Function) --------`--------     clr!MethodDescCallSite::Call_RetArgSlot+0x5 [f:\dd\ndp\clr\src\vm\callhelpers.h @ 423] 
18 00000040`4ddfecc0 00007ff8`3c50bdf7     clr!RunMain+0x266 [f:\dd\ndp\clr\src\vm\assembly.cpp @ 2674] 
19 00000040`4ddfeea0 00007ff8`3c50bcab     clr!Assembly::ExecuteMainMethod+0xb7 [f:\dd\ndp\clr\src\vm\assembly.cpp @ 2795] 
1a 00000040`4ddff190 00007ff8`3c50b5f4     clr!SystemDomain::ExecuteMainMethod+0x643 [f:\dd\ndp\clr\src\vm\appdomain.cpp @ 3755] 
1b 00000040`4ddff790 00007ff8`3c50b34d     clr!ExecuteEXE+0x3f [f:\dd\ndp\clr\src\vm\ceemain.cpp @ 3053] 
1c 00000040`4ddff800 00007ff8`3c50c534     clr!_CorExeMainInternal+0xb2 [f:\dd\ndp\clr\src\vm\ceemain.cpp @ 2887] 
1d 00000040`4ddff890 00007ff8`3cec8c01     clr!_CorExeMain+0x14 [f:\dd\ndp\clr\src\vm\ceemain.cpp @ 2814] 
1e 00000040`4ddff8d0 00007ff8`3d14ac42     mscoreei!_CorExeMain+0x112 [f:\dd\ndp\clr\src\dlls\shim\shim.cpp @ 6425] 
1f 00000040`4ddff930 00007ff8`487f7034     MSCOREE!_CorExeMain_Exported+0x72 [onecore\com\netfx\windowsbuilt\shell_shim\v2api.cpp @ 1223] 
20 00000040`4ddff960 00007ff8`4901d0d1     KERNEL32!BaseThreadInitThunk+0x14 [clientcore\base\win32\client\thread.c @ 64] 
21 00000040`4ddff990 00000000`00000000     ntdll!RtlUserThreadStart+0x21 [minkernel\ntdll\rtlstrt.c @ 1153] 

[felixpelletier]

Still having this issue in 2022. In our case it's when using the command dotnet restore src\Solution.sln (dotnet restore -p:UseSharedCompilation=false src\Solution.sln didn't help)

[neighbour-oldhuang]

我也遇到了这个问题 sad。。 UNHANDLED EXCEPTIONS FROM PROCESS 32204:

2022/9/22 20:42:46 Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Node 5 does not have a provider. 在 Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary2 globalProperties, Dictionary2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Boolean needToValidateProject, String schemaFile, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, TextWriter targetsWriter, Boolean detailedSummary, ISet1 warningsAsErrors, ISet1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler, Boolean interactive, Boolean isolateProjects, GraphBuildOptions graphBuildOptions, Boolean lowPriority, String[] inputResultsCaches, String outputResultsCache) 在 Microsoft.Build.CommandLine.MSBuildApp.Execute(String commandLine) 在 Microsoft.Build.CommandLine.MSBuildApp.Main()

我执行的命令： msbuild.exe -m -t:Rebuild -restore -clp:PerformanceSummary;ErrorsOnly;Summary -flp:LogFile=msbuild.log;Verbosity=normal;PerformanceSummary;ShowTimestamp -p:Configuration=Release;RestorePackagesConfig=true;WarningLevel=1;BuildProjectReferences=true;SolutionDir=n:\jenkins\workspace\Multi-Build-jobv2 n:\jenkins\workspace\Multi-Build-jobv2\S3.IOP.ClientSvr\S3.IOP.ClientSvr.csproj

我的工具版本： 用于 .NET Framework 的 Microsoft (R) 生成引擎版本 16.11.2+f32259642

还没有找到它出现的规律

[pasisavolainen]

Getting this intermittently. Jenkins, linux builder.

[Pipeline] sh (Restoring nuget packages)
+ dotnet restore solution.sln -s http://sourcea/ -s http://sourceb -s http://sourcec
MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.
Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Node 2 does not have a provider.
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInternalError(String message, Object[] args)
   at Microsoft.Build.BackEnd.NodeManager.SendData(Int32 node, INodePacket packet)
   at Microsoft.Build.Execution.BuildManager.PerformSchedulingActions(IEnumerable`1 responses)
   at Microsoft.Build.Execution.BuildManager.PerformSchedulingActions(IEnumerable`1 responses)
   at Microsoft.Build.Execution.BuildManager.HandleNewRequest(Int32 node, BuildRequestBlocker blocker)
   at Microsoft.Build.Execution.BuildManager.ProcessPacket(Int32 node, INodePacket packet)
   at Microsoft.Build.Execution.BuildManager.<>c__DisplayClass76_0.<Microsoft.Build.BackEnd.INodePacketHandler.PacketReceived>b__0()
   at Microsoft.Build.Execution.BuildManager.ProcessWorkQueue(Action action)
--- End of stack trace from previous location ---
   at Microsoft.Build.Execution.BuildManager.EndBuild()
   at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, TextWriter targetsWriter, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler, Boolean interactive, Boolean isolateProjects, GraphBuildOptions graphBuildOptions, Boolean lowPriority, String[] inputResultsCaches, String outputResultsCache)

MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.
Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Node 2 does not have a provider.
   at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, TextWriter targetsWriter, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler, Boolean interactive, Boolean isolateProjects, GraphBuildOptions graphBuildOptions, Boolean lowPriority, String[] inputResultsCaches, String outputResultsCache)
   at Microsoft.Build.CommandLine.MSBuildApp.Execute(String[] commandLine)
Unhandled exception: Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: Node 2 does not have a provider.
   at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, TextWriter targetsWriter, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler, Boolean interactive, Boolean isolateProjects, GraphBuildOptions graphBuildOptions, Boolean lowPriority, String[] inputResultsCaches, String outputResultsCache)
   at Microsoft.Build.CommandLine.MSBuildApp.Execute(String[] commandLine)
   at Microsoft.Build.CommandLine.MSBuildApp.Main(String[] args)
   at Microsoft.DotNet.Cli.Utils.MSBuildForwardingAppWithoutLogging.ExecuteInProc(String[] arguments)
[Pipeline] }

[hknielsen]

We are also running into this error on CI, not sure yet how often, its on a OSX Arm64 matchine;

MSB0001: Internal MSBuild Error: Node 1 does not have a provider.
StackTrace:
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInternalError(String message, Object[] args)
   at Microsoft.Build.BackEnd.NodeManager.SendData(Int32 node, INodePacket packet)
   at Microsoft.Build.Execution.BuildManager.PerformSchedulingActions(IEnumerable`1 responses)
   at Microsoft.Build.Execution.BuildManager.HandleResult(Int32 node, BuildResult result)
   at Microsoft.Build.Execution.BuildManager.ProcessPacket(Int32 node, INodePacket packet)
   at Microsoft.Build.Execution.BuildManager.ProcessWorkQueue(Action action)
--- End of stack trace from previous location ---
   at Microsoft.Build.Execution.BuildManager.EndBuild()
...

[hknielsen]

Ok a coworker got a bit more information of what went wrong with the Node for us:

UNHANDLED EXCEPTIONS FROM PROCESS 7140:
=====================
7/2/2024 1:15:27 PM
Microsoft.Build.Exceptions.InternalLoggerException: The build stopped unexpectedly because of an unexpected logger failure.
 ---> Microsoft.Build.Exceptions.InternalLoggerException: The build stopped unexpectedly because of an unexpected logger failure.
 ---> Microsoft.Build.Exceptions.InvalidProjectFileException: The imported project file "xxx.props" could not be loaded. The process cannot access the file 'xxx.props" ' because it is being used by another process.  xxx\Sdk.props
   at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject(String errorSubCategoryResourceName, IElementLocation elementLocation, String resourceName, Object[] args)
   at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject[T1,T2](IElementLocation elementLocation, String resourceName, T1 arg0, T2 arg1)
   at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImportsFromUnescapedImportExpression(String directoryOfImportingFile, ProjectImportElement importElement, String unescapedExpression, Boolean throwOnFileNotExistsError, List`1& imports)
   at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImportsFromUnescapedImportExpressionConditioned(String directoryOfImportingFile, ProjectImportElement importElement, List`1& projects, SdkResult& sdkResult)
   at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImports(String directoryOfImportingFile, ProjectImportElement importElement, SdkResult& sdkResult)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.EvaluateImportElement(String directoryOfImportingFile, ProjectImportElement importElement)
   at Microsoft.Build.Evaluation.Evaluator`4.PerformDepthFirstPass(ProjectRootElement currentProjectOrImport)
   at Microsoft.Build.Evaluation.Evaluator`4.Evaluate()
   at Microsoft.Build.Evaluation.Evaluator`4.Evaluate(IEvaluatorData`4 data, Project project, ProjectRootElement root, 

[danmoseley]

@hknielsen I think that's unrelated (unless you're getting the other error as well). I suggest opening another issue. That should not happen.