benaadams / Ben.BlockingDetector

Blocking Detection for ASP.NET Core
Apache License 2.0
731 stars 35 forks source link

A lot of warnings about framework code #5

Open guylando opened 6 years ago

guylando commented 6 years ago

Great package, thanks for making it!

We want to solve our blocking problems but there are too many warnings regarding dotnet framework code and it makes too much noise to be able to use the package to detect problems in our own code. Is the package supposed to throw those warning? Here are some of them:

1)

warn: Ben.Diagnostics.BlockingMonitor[6] Blocking method has been invoked and blocked, this can lead to threadpool starvation. at Microsoft.Net.Http.Headers.DateTimeFormatter.ToRfc1123String(DateTimeOffset dateTime, Boolean quoted) at Microsoft.AspNetCore.Http.HeaderDictionaryTypeExtensions.SetDate(IHeaderDictionary headers, String name, Nullable1 value) at Microsoft.AspNetCore.Http.Headers.ResponseHeaders.set_LastModified(Nullable1 value) at Microsoft.AspNetCore.StaticFiles.StaticFileContext.ApplyResponseHeaders(Int32 statusCode) at Microsoft.AspNetCore.StaticFiles.StaticFileContext.<SendAsync>d__49.MoveNext() at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.AspNetCore.StaticFiles.StaticFileContext.SendAsync() at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.<Invoke>d__3.MoveNext() at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.AspNetCore.Diagnostics.StatusCodePagesMiddleware.Invoke(HttpContext context) at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.<Invoke>d__7.MoveNext() at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context) at Ben.Diagnostics.BlockingDetectionMiddleware.<Invoke>d__6.MoveNext() at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine) at Ben.Diagnostics.BlockingDetectionMiddleware.Invoke(HttpContext httpContext) ...

2) warn: Ben.Diagnostics.BlockingMonitor[6] Blocking method has been invoked and blocked, this can lead to threadpool starvation. at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.CodeAnalysis.CSharp.MethodCompiler.WaitForWorkers() at Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileMethodBodies(CSharpCompilation compilation, PEModuleBuilder moduleBeingBuiltOpt, Boolean emittingPdb, Boolean emitTestCoverageData, Boolean hasDeclarationErrors, DiagnosticBag diagnostics, Predicate1 filterOpt, CancellationToken cancellationToken) at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.CompileMethods(CommonPEModuleBuilder moduleBuilder, Boolean emittingPdb, Boolean emitMetadataOnly, Boolean emitTestCoverageData, DiagnosticBag diagnostics, Predicate1 filterOpt, CancellationToken cancellationToken) at Microsoft.CodeAnalysis.Compilation.Emit(Stream peStream, Stream metadataPEStream, Stream pdbStream, Stream xmlDocumentationStream, Stream win32Resources, IEnumerable1 manifestResources, EmitOptions options, IMethodSymbol debugEntryPoint, Stream sourceLinkStream, IEnumerable1 embeddedTexts, CompilationTestData testData, CancellationToken cancellationToken) at Microsoft.CodeAnalysis.Compilation.Emit(Stream peStream, Stream pdbStream, Stream xmlDocumentationStream, Stream win32Resources, IEnumerable1 manifestResources, EmitOptions options, IMethodSymbol debugEntryPoint, Stream sourceLinkStream, IEnumerable1 embeddedTexts, Stream metadataPEStream, CancellationToken cancellationToken) at Microsoft.AspNetCore.Mvc.Razor.Internal.RazorViewCompiler.CompileAndEmit(RazorCodeDocument codeDocument, String generatedCode) at Microsoft.AspNetCore.Mvc.Razor.Internal.RazorViewCompiler.CompileAndEmit(String relativePath) at Microsoft.AspNetCore.Mvc.Razor.Internal.RazorViewCompiler.CreateCacheEntry(String normalizedPath) at Microsoft.AspNetCore.Mvc.Razor.Internal.RazorViewCompiler.CompileAsync(String relativePath) at Microsoft.AspNetCore.Mvc.Razor.Internal.DefaultRazorPageFactoryProvider.CreateFactory(String relativePath) at Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.CreateCacheResult(HashSet1 expirationTokens, String relativePath, Boolean isMainPage) .... `

3)

warn: Ben.Diagnostics.BlockingMonitor[6] Blocking method has been invoked and blocked, this can lead to threadpool starvation. at System.ModuleHandle.ResolveType(RuntimeModule module, Int32 typeToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount, ObjectHandleOnStack type) at System.ModuleHandle.ResolveTypeHandleInternal(RuntimeModule module, Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext) at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments) at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(CustomAttributeRecord caRecord, MetadataImport scope, Assembly& lastAptcaOkAssembly, RuntimeModule decoratedModule, MetadataToken decoratedToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, Object[] attributes, IList derivedAttributes, RuntimeType& attributeType, IRuntimeMethodInfo& ctor, Boolean& ctorHasParameters, Boolean& isVarArg) at System.Reflection.CustomAttribute.IsCustomAttributeDefined(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Int32 attributeCtorToken, Boolean mustBeInheritable) at System.Reflection.CustomAttribute.IsDefined(RuntimeType type, RuntimeType caType, Boolean inherit) at Microsoft.AspNetCore.Mvc.Controllers.ControllerFeatureProvider.IsController(TypeInfo typeInfo) at Microsoft.AspNetCore.Mvc.Controllers.ControllerFeatureProvider.PopulateFeature(IEnumerable1 parts, ControllerFeature feature) at Microsoft.AspNetCore.Mvc.ApplicationParts.ApplicationPartManager.PopulateFeature[TFeature](TFeature feature) `

benaadams commented 6 years ago

That's pretty interesting... /cc @davidfowl

guylando commented 6 years ago

@benaadams any news maybe?

AlonCG commented 6 years ago

I know this thread is a few months old, but I have the same issue as the OP.

Just by debugging my application, starting on a "Ping" page (the one I use for azure load balancing) ... I end up with 61k lines of output warnings for "Ben.Diagnostics.BlockingMonitor". Out of these 61k lines, there are ~100 lines of issues that I need to actually focus on.

This is a nifty tool (so thanks for that), though I presume it will no longer be actively developed? I am not sure I have the understanding to be a contributor, but it would be great to be able to cut out all the excessive framework noise.

benaadams commented 6 years ago

They aren't incorrect :)

Can you post some of the blocking traces and maybe we can get them resolved in the framework?

Also are they one offs, e.g. during startup vs continuous (the latter being more problematic)

AlonCG commented 6 years ago

Ok ... so if these are all blocking calls, then I will take a closer (more thorough) look at them :-)

I was just hoping to throw out things that were not related to my code specifically. I do see a lot of AppInsights and DeveloperExceptionPageMiddleware errors, so maybe I have something wrong in the startup and will try to trim them down.

Thanks again!

guylando commented 6 years ago

@benaadams Would appreciate an options flag allowing to choose if the warnings will include framework\nuggets code or not

daharmon commented 6 years ago

I have the same issue. Even asp-append-version for css and javascript files in layout pages causing this warning, not to mention just rendering the razor file in the first place. It seems like all the asp.net razor code is blocking. It seems like it generates thousands of lines of warnings for each page load, which makes it very difficult to figure out the real issue. I have an .net core site that seems to be hanging about once a day, sometimes more with extremely high traffic. I have everything await/async so I don't think I have blocking calls but I was hoping this tool could help me confirm.

benaadams commented 6 years ago

Might be overly sensitive; e.g. it will trigger the warning for locks that block

Should be able to add a sensitively flag

davidfowl commented 6 years ago

Should be able to add a sensitively flag

Only do Task.Wait and Result by default, the other locks should be opt-in (aggressive mode).

hempels commented 5 years ago

This is a pretty cool idea, but it ends up being pretty useless for the reasons mentioned above. Too bad. Would be lovely to have a way to find misbehaving user code without having to fight through warnings about framework libraries.

guylando commented 5 years ago

Maybe an option to blacklist or whitelist assemblies or namespaces where errors are reported? This would allow a process of filtering until the desired error is found.