search

dotnet / aspnetcore

ASP.NET Core is a cross-platform .NET framework for building modern cloud-based web applications on Windows, Mac, or Linux.
https://asp.net
MIT License
35.59k stars 10.06k forks source link

SignalR Hub Method Invocation Activities are not Recorded while auto Instrumentation is Configured. #58861

Closed Javidleo closed 2 weeks ago

Javidleo commented 3 weeks ago

Is there an existing issue for this?

Describe the bug

Hi, I used open telemetry in .NET for my microservice based solution, everything works fine, but there is a problem with SignalR working correctly as expected with Activities,

My current version of .NET is 8. Open Telemetry Packages are in 1.9.0

with apis, everything is okey and works fine, but when it comes to call Hub Methods, Activity.Current.Recorded returns false and I cannot receive any activities in my Grafana dashboard, this is the full activity json data 

{
    "Status": 0,
    "StatusDescription": null,
    "HasRemoteParent": true,
    "Kind": 1,
    "OperationName": "Microsoft.AspNetCore.Hosting.HttpRequestIn",
    "DisplayName": "Microsoft.AspNetCore.Hosting.HttpRequestIn",
    "Source": {
        "Name": "Microsoft.AspNetCore",
        "Version": ""
    },
    "Parent": null,
    "Duration": "00:00:00",
    "StartTimeUtc": "2024-11-11T05:16:57.4758477Z",
    "Id": "00-ff13afb3bbcea2102cfd03c71e854124-18967c1f38c19913-00",
    "ParentId": "00-ff13afb3bbcea2102cfd03c71e854124-292907f66e8270ef-00",
    "RootId": "ff13afb3bbcea2102cfd03c71e854124",
    "Tags": [
        {
            "Key": "http.long_running",
            "Value": "true"
        }
    ],
    "TagObjects": [
        {
            "Key": "http.long_running",
            "Value": "true"
        }
    ],
    "Events": [],
    "Links": [],
    "Baggage": [],
    "Context": {
        "TraceId": {},
        "SpanId": {},
        "TraceFlags": 0,
        "TraceState": null,
        "IsRemote": false
    },
    "TraceStateString": null,
    "SpanId": {},
    "TraceId": {},
    "Recorded": false,
    "IsAllDataRequested": false,
    "ActivityTraceFlags": 0,
    "ParentSpanId": {},
    "IsStopped": false,
    "IdFormat": 2
}

meanwhile the Metrics are not working either. I can see them in console when Im using console exporter, but cannot found them in granfa.

other possible issues you may mention: I already checked the configuration to ensure everything works correctly, because all of my services are using a global configuration like this : 

public static IServiceCollection AddObservabilityDefaults(this IServiceCollection services,
    IWebHostEnvironment env,
    string serviceName = "test-service",
    string[] additionalSources = null!,
    string[] additionalMetrics = null!,
    string[] ignoreCases = null!)
{
    services.AddOpenTelemetry()
        .WithMetrics(metrics =>
        {
            metrics.AddRuntimeInstrumentation()
                   .AddProcessInstrumentation()
                   .AddBuiltInMeters()
                   .AddAspNetCoreInstrumentation()
                   .AddHttpClientInstrumentation()
                   //.AddConsoleExporter()
                   .AddPrometheusExporter()
                   .AddConsoleExporter(); // /metrics

            if (additionalMetrics.IsEmpty() is false)
                metrics.AddMeter(additionalMetrics);
        })
        .WithTracing(tracing =>
        {
            //if (env.IsDevelopment())
            //{
            //    // We want to view all traces in development
            //    tracing.SetSampler(new AlwaysOnSampler());
            //}
            tracing.AddSource(EventBusRabbitMQ.EventBusRabbitMQ.Activity_Source_Name);

            tracing.SetResourceBuilder(ResourceBuilder.CreateDefault().AddService(serviceName));

            if (additionalSources.NotEmpty())
                tracing.AddSource(additionalSources);

            tracing.AddAspNetCoreInstrumentation(i =>
            {
                i.Filter = (httpContext) =>
                {
                    var path = httpContext.Request.Path.Value;

                    if (ignoreCases != null)
                        DefaultIgnoreCases.AddRange(ignoreCases);

                    foreach (var ignoreCase in DefaultIgnoreCases)
                    {
                        if (path!.StartsWith(ignoreCase, StringComparison.OrdinalIgnoreCase))
                            return false;
                    }
                    return true;
                };
            })
            .AddHttpClientInstrumentation(option =>
            {
                option.FilterHttpRequestMessage =
                    (req) =>
                        req.RequestUri!.AbsoluteUri.Contains("elastic") == false;

                option.FilterHttpWebRequest =
                    (webreq) =>
                        webreq.RequestUri.AbsoluteUri.Contains("elastic") == false;
            })
            .AddOtlpExporter(i =>
            {
                // working with docker compose.
                i.Endpoint = new Uri("http://collector:4317");
            })
            .AddConsoleExporter();
        });

    return services;
}

this is all I use, questions:

can anyone clarify these problems for me? thanks.

Expected Behavior

No response

Steps To Reproduce

No response

Exceptions (if any)

No response

.NET Version

No response

Anything else?

No response

davidfowl commented 2 weeks ago

I searched a lot in internet to find anything related to SignalR Open Telemetry Configuration, I also use the above method after the SignalR service registration in Startup, why there is nothing really exist for this topic? checked this link but its for .NET 9 : https://learn.microsoft.com/en-us/aspnet/core/release-notes/aspnetcore-9.0?view=aspnetcore-8.0

It's in .NET 9.

why its not possible to create manual activities ? I use rabbit mq as mesage broker but when an event published from a Hub Method, the Rabbit MQ Activity Source Returns null when calling StartActivity,

@noahfalk do we have end to end docs for using activities?

Javidleo commented 2 weeks ago

I searched a lot in internet to find anything related to SignalR Open Telemetry Configuration, I also use the above method after the SignalR service registration in Startup, why there is nothing really exist for this topic? checked this link but its for .NET 9: https://learn.microsoft.com/en-us/aspnet/core/release-notes/aspnetcore-9.0?view=aspnetcore-8.0

It's in .NET 9.

so we don't have it in .NET 8?

davidfowl commented 2 weeks ago

so we don't have it in .NET 8?

No. There's this package that someone built that supports .NET 8 though https://www.nuget.org/packages/AspNetCore.SignalR.OpenTelemetry

Javidleo commented 2 weeks ago

I searched a lot in internet to find anything related to SignalR Open Telemetry Configuration, I also use the above method after the SignalR service registration in Startup, why there is nothing really exist for this topic? checked this link but its for .NET 9 : https://learn.microsoft.com/en-us/aspnet/core/release-notes/aspnetcore-9.0?view=aspnetcore-8.0

It's in .NET 9.

why its not possible to create manual activities ? I use rabbit mq as mesage broker but when an event published from a Hub Method, the Rabbit MQ Activity Source Returns null when calling StartActivity,

@noahfalk do we have end to end docs for using activities?

I also need to mention that this problem also happens for when I use signalR. everything works fine with http endpoints.

Javidleo commented 2 weeks ago

so we don't have it in .NET 8?

No. There's this package that someone built that supports .NET 8 though https://www.nuget.org/packages/AspNetCore.SignalR.OpenTelemetry

thanks I give it a try.

Javidleo commented 2 weeks ago

I close the Issue for now, thanks for your time 😄

noahfalk commented 2 weeks ago

@noahfalk do we have end to end docs for using activities?

It won't answer specifics about SignalR, but these are the docs: .NET overview of distributed tracing .NET guide on setting up OpenTelemetry (incudes distributed tracing) OpenTelemetry-dotnet repo's distributed tracing getting started guide