search

djluck / prometheus-net.DotNetRuntime

Exposes .NET core runtime metrics (GC, JIT, lock contention, thread pool) using the prometheus-net package
MIT License
357 stars 85 forks source link

NullReferenceException thrown when GcsStats CaptureLevel is set to Verbose and app is running locally in Debug mode #65

Closed AnnaGorozia closed 2 years ago

AnnaGorozia commented 2 years ago

When we run our application locally in Debug mode, in runtime it throws NullReferenceException. It only happens when we configure DotNetRuntimeStats for GcsStats with CaptureLevel.Verbose. If we set CaptureLevel.Informational exception is not thrown.

I couldn't determine exact source of exception. Providing stack trace which I currently get:

System.NullReferenceException: 'Object reference not set to an instance of an object.'
This exception was originally thrown at this call stack:
    System.Diagnostics.Tracing.EventPipePayloadDecoder.DecodePayload(ref System.Diagnostics.Tracing.EventSource.EventMetadata, System.ReadOnlySpan<byte>)
    System.Diagnostics.Tracing.NativeRuntimeEventSource.ProcessEvent(uint, uint, System.DateTime, System.Guid, System.Guid, System.ReadOnlySpan<byte>)
    System.Diagnostics.Tracing.EventPipeEventDispatcher.DispatchEventsToEventListeners()
    System.Threading.Tasks.Task.InnerInvoke()
    System.Threading.Tasks.Task..cctor.AnonymousMethod__272_0(object)
    System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, object)
    System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

image

I created sample app which shows similar behavior. You can check it out here: https://github.com/AnnaGorozia/PrometheusSample

smhoff commented 2 years ago

We are seeing the same thing. I'd just add that it only occurs for us when we convert our application to use .Net 6. It works fine in .Net 5.

djluck commented 2 years ago

Hmm interesting. Going by the stack trace alone, it appears to be an issue with .NET 6 itself, not my code. Will try and reproduce myself.

djluck commented 2 years ago

I'm unable to reproduce- I suspect this is an issue with Visual Studio (although I cannot confirm this as I don't have a copy). When running in both the debug configuration and in Rider, the sample application didn't fail for me. Also, all the preexisting verbose GC event listener tests that target .net 6.0 are working as expected for me.

From memory, visual studio starts telemetry sessions when debugging to collect and display useful information (such as GCs)- I believe there is a conflict going on here.

djluck commented 2 years ago

As a workaround, for now I can suggest using Debugger.IsAttached to detect when a debugger (such as VS) is detected- you can then not enable any verbose-level collection. It wouldn't be appropriate to include this in the library as it seems that this is a VS-specific issue.

sandersaares commented 2 years ago

This is now fixed in .NET 7 and backported into next .NET 6 servicing release. 🥳

djluck commented 2 years ago

Thanks @sandersaares! Closing for now.