[BUG] {AspNetCoreDiagnosticListener} Failed capturing DiagnosticSource event (System.NullReferenceException)

[OlegUfaev]

APM Agent version

Elastic.Apm.AspNetCore v1.28.0

Environment

Operating system and version: Windows 10

.NET Framework/Core name and version (e.g. .NET 4.6.2, NET Core 3.1.100) : .NET 8

Application Target Framework(s) (e.g. net462, netcoreapp3.1): .net8.0

Describe the bug

If an ASP.NET Core application processes a HTTP request with distributed tracing headers (traceparent and baggage) AND an exception occurs in the controller, then the Elastic.Apm logger writes the following log entry.

fail: Elastic.Apm[0]
      {AspNetCoreDiagnosticListener} Failed capturing DiagnosticSource event
      System.NullReferenceException: Object reference not set to an instance of an object.
         at Elastic.Apm.Model.Error.CheckAndCaptureBaggage() in C:\Users\oufaev\source\repos\apm-agent-dotnet\src\Elastic.Apm\Model\Error.cs:line 77
         at Elastic.Apm.Model.Error..ctor(Transaction transaction, String parentId, IApmLogger loggerArg, Dictionary`2 labels) in C:\Users\oufaev\source\repos\apm-agent-dotnet\src\Elastic.Apm\Model\Error.cs:line 58
         at Elastic.Apm.Model.Error..ctor(CapturedException capturedException, Transaction transaction, String parentId, IApmLogger loggerArg, Dictionary`2 labels) in C:\Users\oufaev\source\repos\apm-agent-dotnet\src\Elastic.Apm\Model\Error.cs:line 25
         at Elastic.Apm.Model.ExecutionSegmentCommon.CaptureException(Exception exception, IApmLogger logger, IPayloadSender payloadSender, IExecutionSegment executionSegment, IConfigurationReader configurationReader, Transaction transaction, IApmServerInfo apmServerInfo, String culprit, Boolean isHandled, String parentId, Dictionary`2 labels) in C:\Users\oufaev\source\repos\apm-agent-dotnet\src\Elastic.Apm\Model\ExecutionSegmentCommon.cs:line 203
         at Elastic.Apm.Model.Transaction.CaptureException(Exception exception, String culprit, Boolean isHandled, String parentId, Dictionary`2 labels) in C:\Users\oufaev\source\repos\apm-agent-dotnet\src\Elastic.Apm\Model\Transaction.cs:line 779
         at Elastic.Apm.AspNetCore.DiagnosticListener.AspNetCoreDiagnosticListener.HandleOnNext(KeyValuePair`2 kv) in C:\Users\oufaev\source\repos\apm-agent-dotnet\src\integrations\Elastic.Apm.AspNetCore\DiagnosticListener\AspNetCoreDiagnosticListener.cs:line 89
         at Elastic.Apm.DiagnosticListeners.DiagnosticListenerBase.OnNext(KeyValuePair`2 kv) in C:\Users\oufaev\source\repos\apm-agent-dotnet\src\Elastic.Apm\DiagnosticListeners\DiagnosticListenerBase.cs:line 53

To Reproduce

Steps to reproduce the behavior:

1. Get source code: https://github.com/elastic/apm-agent-dotnet/tree/v1.28.0
2. Run project WebApiExample
3. Then do HTTP request. To reproduce the problem it is really important to send the baggage header, and in the traceparent header the field with the sampled flag should be 00.

   GET https://localhost:64661/Error
   traceparent: 00-8461f428892542f2611dbc136acd91d9-bbdf919f035206a7-00
   baggage: key1=value1
   accept: application/json
   content-type: application/json
   host: localhost:64661

4. See error in log

Expected behavior

The System.NullReferenceException error does not occur.

Actual behavior

System.NullReferenceException error occurs.

The problem occurs in this method: Error.cs

        private void CheckAndCaptureBaggage()
        {
            if (Activity.Current == null || !Activity.Current.Baggage.Any())
                return;

            foreach (var baggage in Activity.Current.Baggage)
            {
                if (!WildcardMatcher.IsAnyMatch(Configuration.BaggageToAttach, baggage.Key))
                    continue;

                var newKey = $"baggage.{baggage.Key}";
                var labels = Context.InternalLabels.Value;   // Here Context is null, so the error occurs
                labels[newKey] = baggage.Value;
            }
        }

[Mpdreamz]

Thanks for reporting this @OlegUfaev, this sounds like a nasty issue. This didn't make the cutoff for the several bug fix releases we did this week but I put this at the top of my queue on Monday.

[Mpdreamz]

Opened https://github.com/elastic/apm-agent-dotnet/pull/2427 to address this.