dotnet / msbuild

The Microsoft Build Engine (MSBuild) is the build platform for .NET and Visual Studio.
https://docs.microsoft.com/visualstudio/msbuild/msbuild
MIT License
5.21k stars 1.35k forks source link

Build fails when try to process the bundleconfig.json #3362

Closed KostaVlev closed 4 years ago

KostaVlev commented 6 years ago

We have CI setup in VSTS and all of a sudden. Even on previously successful builds.

The build started failing when it tries to process the bundleconfig.json I can confirm we don't have any changes in that file and previous builds were successful. The build fails at the same point even if I try to rebuild branches where it was successful.

SYSTEM CAPABILITIES Shows information about the capabilities provided by this host Capability name Capability value Agent.Name Hosted VS2017 2 Agent.Version 2.134.2 AndroidSDK C:\Program Files (x86)\Android\android-sdk ant C:\java\ant\apache-ant-1.9.7 AzurePS 3.6.0 Cmd C:\Windows\system32\cmd.exe curl C:\Program Files\Git\mingw64\bin\curl.exe DotNetFramework C:\Windows\Microsoft.NET\Framework64\v4.0.30319 GTK_BASEPATH C:\Program Files (x86)\GtkSharp\2.12\ java C:\Program Files (x86)\Java\jdk1.8.0_112 jdk C:\Program Files (x86)\Java\jdk1.8.0_112 jdk_8 C:\Program Files (x86)\Java\jdk1.8.0_112 maven C:\java\maven\apache-maven-3.2.2 MSBuild C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\ node.js C:\Program Files\nodejs\node.exe npm C:\Program Files\nodejs\npm.cmd SqlPackage C:\Program Files\Microsoft SQL Server\140\DAC\bin\SqlPackage.exe VisualStudio C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\ VSTest C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow Xamarin.Android 7.2.0.7

Error logs Bundler: Begin processing bundleconfig.json Build FAILED. ....

MSBUILD : error MSB1025: An internal failure occurred while running MSBuild. Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: ContextID 22 should have been in the ID-to-project file mapping but wasn't! at Microsoft.Build.Shared.ErrorUtilities.ThrowInternalError(String message, Object[] args) at Microsoft.Build.BackEnd.Logging.LoggingService.GetAndVerifyProjectFileFromContext(BuildEventContext context) at Microsoft.Build.BackEnd.Logging.LoggingService.LogBuildEvent(BuildEventArgs buildEvent) at Microsoft.Build.BackEnd.TaskHost.LogErrorEvent(BuildErrorEventArgs e) at Microsoft.Build.Utilities.TaskLoggingHelper.LogError(String subcategory, String errorCode, String helpKeyword, String file, Int32 lineNumber, Int32 columnNumber, Int32 endLineNumber, Int32 endColumnNumber, String message, Object[] messageArgs) at BundlerMinifier.BundlerBuildTask.BundleMinifier_ErrorMinifyingFile(Object sender, MinifyFileEventArgs e) at BundlerMinifier.BundleMinifier.MinifyBundle(Bundle bundle) at BundlerMinifier.BundleFileProcessor.ProcessBundle(String baseFolder, Bundle bundle) at BundlerMinifier.BundleFileProcessor.Process(String fileName, IEnumerable1 bundles) at BundlerMinifier.BundlerBuildTask.Execute() at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) at Microsoft.Build.BackEnd.TaskBuilder.InitializeAndExecuteTask(TaskLoggingContext taskLoggingContext, ItemBucket bucket, IDictionary2 taskIdentityParameters, TaskHost taskHost, TaskExecutionMode howToExecuteTask) at Microsoft.Build.BackEnd.TaskBuilder.ExecuteBucket(TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask, Dictionary2 lookupHash) at Microsoft.Build.BackEnd.TaskBuilder.ExecuteTask(TaskExecutionMode mode, Lookup lookup) at Microsoft.Build.BackEnd.TaskBuilder.ExecuteTask(TargetLoggingContext loggingContext, BuildRequestEntry requestEntry, ITargetBuilderCallback targetBuilderCallback, ProjectTargetInstanceChild taskInstance, TaskExecutionMode mode, Lookup inferLookup, Lookup executeLookup, CancellationToken cancellationToken) at Microsoft.Build.BackEnd.TargetEntry.ProcessBucket(ITaskBuilder taskBuilder, TargetLoggingContext targetLoggingContext, TaskExecutionMode mode, Lookup lookupForInference, Lookup lookupForExecution) at Microsoft.Build.BackEnd.TargetEntry.ExecuteTarget(ITaskBuilder taskBuilder, BuildRequestEntry requestEntry, ProjectLoggingContext projectLoggingContext, CancellationToken cancellationToken) at Microsoft.Build.BackEnd.TargetBuilder.ProcessTargetStack(ITaskBuilder taskBuilder) at Microsoft.Build.BackEnd.TargetBuilder.BuildTargets(ProjectLoggingContext loggingContext, BuildRequestEntry entry, IRequestBuilderCallback callback, String[] targetNames, Lookup baseLookup, CancellationToken cancellationToken) at Microsoft.Build.BackEnd.RequestBuilder.BuildProject() at Microsoft.Build.BackEnd.RequestBuilder.BuildAndReport() at Microsoft.Build.BackEnd.RequestBuilder.RequestThreadProc(Boolean setThreadParameters) MSBUILD : error MSB1025: An internal failure occurred while running MSBuild. Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: ContextID 22 should have been in the ID-to-project file mapping but wasn't! at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary2 globalProperties, Dictionary2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, Boolean detailedSummary, ISet1 warningsAsErrors, ISet1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler) at Microsoft.Build.CommandLine.MSBuildApp.Execute(String[] commandLine) Unhandled Exception: Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: ContextID 22 should have been in the ID-to-project file mapping but wasn't! at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary2 globalProperties, Dictionary2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, Boolean detailedSummary, ISet1 warningsAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler) at Microsoft.Build.CommandLine.MSBuildApp.Execute(String[] commandLine) at Microsoft.Build.CommandLine.MSBuildApp.Main(String[] args)

Any suggestions what went wrong!?

mattduguid commented 6 years ago

We have also just started seeing the same issue, can compile locally using VS but not via VSTS, still looking into it and we have a Microsoft call open but no idea of root cause(s) yet,

2018-06-04T14:03:44.9818431Z Bundler: Begin processing bundleconfig.json 2018-06-04T14:03:46.2516820Z 2018-06-04T14:03:46.2519207Z Build FAILED. 2018-06-04T14:03:46.2523794Z 0 Warning(s) 2018-06-04T14:03:46.2533969Z 0 Error(s) 2018-06-04T14:03:46.2534165Z 2018-06-04T14:03:46.2536071Z Time Elapsed 00:00:06.23 2018-06-04T14:03:46.2581471Z MSBUILD : error MSB1025: An internal failure occurred while running MSBuild. 2018-06-04T14:03:46.2581909Z Microsoft.Build.Shared.InternalErrorException: MSB0001: Internal MSBuild Error: ContextID 22 should have been in the ID-to-project file mapping but wasn't! 2018-06-04T14:03:46.2582301Z at Microsoft.Build.Shared.ErrorUtilities.ThrowInternalError(String message, Object[] args) 2018-06-04T14:03:46.2582433Z at Microsoft.Build.BackEnd.Logging.LoggingService.GetAndVerifyProjectFileFromContext(BuildEventContext context) 2018-06-04T14:03:46.2582580Z at Microsoft.Build.BackEnd.Logging.LoggingService.LogBuildEvent(BuildEventArgs buildEvent) 2018-06-04T14:03:46.2583537Z at Microsoft.Build.BackEnd.TaskHost.LogErrorEvent(BuildErrorEventArgs e) ...

...
KostaVlev commented 6 years ago

It turned out the problem is the BundlerMinifier NuGet package. After switching to BundlerMinifier.Core version everything builds. I don’t know why that happened all of a sudden.

cstruter commented 6 years ago

@KostaVlev Just double check on your side that it actually still minifies resources, I switched to the BundlerMinifier.Core version, everything seemed fine, until I noticed minification actually stopped working.

The .Core project is used for ad hoc execution of bundling and minification.

E.g dotnet bundle

https://docs.microsoft.com/en-us/aspnet/core/client-side/bundling-and-minification?view=aspnetcore-2.1&tabs=visual-studio%2Caspnetcore2x

cstruter commented 6 years ago

Just as a note, all of this started happening when upgrading to .net core 2.1, when I reverted back to core 2.0 everything worked again.

mbp commented 6 years ago

@cstruter did you find a fix for this? Started happening for us

cstruter commented 6 years ago

@mbp Did you upgrade to the Core version? I ended up including the dotnet bundle command in our vsts build definition.

KostaVlev commented 6 years ago

@cstruter I can confirm it minifies the files. Our project is .Net Core 1 but I don't think that really matters

mbp commented 6 years ago

@cstruter not yet, but that might be the solution we will go for.

rainersigwald commented 6 years ago

Reopening because even if the right fix is to refer to a different package, MSBuild shouldn't crash this way.

I tried to reproduce this with a dotnet new mvc project and dotnet add package BuildBundlerMinifier, but couldn't. If you're seeing this, can you provide detailed repro information, including dotnet --info version?

mbp commented 6 years ago

@rainersigwald I have not been able to repro in a small project. In my case on a VSTS build agent in a larger solution, and if you retry the build it succeeds. This is the dotnet --info for the build server:

.NET Core SDK (reflecting any global.json):
 Version:   2.1.400
 Commit:    8642e60a0f

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.14393
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.1.400\

Host (useful for support):
  Version: 2.1.2
  Commit:  811c3ce6c0

.NET Core SDKs installed:
  1.1.5 [C:\Program Files\dotnet\sdk]
  2.0.2 [C:\Program Files\dotnet\sdk]
  2.0.3 [C:\Program Files\dotnet\sdk]
  2.1.2 [C:\Program Files\dotnet\sdk]
  2.1.4 [C:\Program Files\dotnet\sdk]
  2.1.100 [C:\Program Files\dotnet\sdk]
  2.1.101 [C:\Program Files\dotnet\sdk]
  2.1.102 [C:\Program Files\dotnet\sdk]
  2.1.103 [C:\Program Files\dotnet\sdk]
  2.1.104 [C:\Program Files\dotnet\sdk]
  2.1.200 [C:\Program Files\dotnet\sdk]
  2.1.201 [C:\Program Files\dotnet\sdk]
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.400 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 1.0.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download
andrew-gaston commented 6 years ago

I had a similar error message and googled it "ContextID 33 should have been in the ID-to-project file mapping but wasn't!" My project is a razor pages project running .net core 2.1, building locally using VS Code.

What seemed to be the issue for me is that in my bundleconfig.json file, my JS input files looked like this: "inputFiles": [ "wwwroot/js/site.js", "wwwroot/js/dropzone.js" ],

The problem with this is that site.js relies on dropzone.js, so changing the order to this: "inputFiles": [ "wwwroot/js/dropzone.js", "wwwroot/js/site.js" ],

Seems to have solved the problem for me. Make sure you put dependencies first. Not sure if this is the right place to put this.

mattstermiller commented 5 years ago

As suggested in madskristensen/BundlerMinifier#365, setting the environment variable MSBUILDDISABLENODEREUSE to 1 is a workaround for this issue.

chaim1221 commented 5 years ago

Unfortunately just wasted a good two hours on this issue, any chance it's going to be fixed soon? Mac OS X,

~:$ dotnet --version
2.2.103

Sorry and thanks!

qcc-na commented 4 years ago

Getting this on 2.2.7

kajetan-kazimierczak commented 4 years ago

Started getting this error on .net core 3.0. Worked for me before.

rainersigwald commented 4 years ago

We tracked down one cause of an error like this: NuGet was holding onto a TaskLoggingHelper in a static field past its expected lifetime (NuGet/Home#8688). The reporting in MSBuild could be improved (#4828).

It's possible that other tasks have similar problems. As before, we'd love to see a repro for this problem to help track it down.

@qcc-na @kajetan-kazimierczak can you elaborate on your problems, please?

kajetan-kazimierczak commented 4 years ago

@rainersigwald I started getting this error on Azure DevOps after I converted my project from .net core 2.2 to .net core 3.0. It's not consistent. Usually the build works.

It looks like it can be triggered by doing "dotnet clean" and then "dotnet build" (not every time, but often). After it successfully builds the project once, subsequent builds seem to mostly work. I added the clean step to my DevOps build after upgrading to core 3.0, so it was maybe therefore I see the error more often now.

I'll try to create a sample to repro this and post it here later.

rainersigwald commented 4 years ago

If you're building using dotnet build, I think the most relevant followup question is "what NuGet package references are in the project?"

shaulbehr commented 4 years ago

It turned out the problem is the BundlerMinifier NuGet package. After switching to BundlerMinifier.Core version everything builds. I don’t know why that happened all of a sudden.

Thank you! This solved it for me!

rainersigwald commented 4 years ago

Sounds like that's the root cause for several folks. If anyone sees this in a project that doesn't use BundlerMinifier, please open a new issue with a repro and we can help track down the problem.

esteban-gs commented 4 years ago

Passing MSBUILDDISABLENODEREUSE=1 as a user secret solved the issue in development. On Ubuntu 18.04.02 with .NetCore 2.2

DanielGlos commented 3 years ago

Passing MSBUILDDISABLENODEREUSE=1 as a user secret solved the issue in development. On Ubuntu 18.04.02 with .NetCore 2.2

Could you please provide some guidelines where to include this variable? I tried adding it like this: variables: MSBUILDDISABLENODEREUSE: 1

inside my pipeline YAML file but it didn't help

chaim1221 commented 3 years ago

I tried adding it like this: variables: MSBUILDDISABLENODEREUSE: 1

inside my pipeline YAML file but it didn't help

Usually I'd ignore this but a couple of notes:

  1. This issue is closed.
  2. This issue applies to a version of MVC that is no longer supported.
  3. There is no ambiguity in a typical developer's mind about what "environment variable" means. If you are having trouble with setting environment variables, rtdd.
  4. Re: YAML. We don't know what your pipeline is; managing your build is up to you. Consult the docs for your build service.
esteban-gs commented 3 years ago

Passing MSBUILDDISABLENODEREUSE=1 as a user secret solved the issue in development. On Ubuntu 18.04.02 with .NetCore 2.2

@DanielGlos , I actually passed it as an environment variable, not a user secret: export MSBUILDDISABLENODEREUSE=1