grafana / grafana-opentelemetry-dotnet

Grafana's distribution of OpenTelemetry instrumentation for .NET
Apache License 2.0
31 stars 5 forks source link

Resource config aren't working on azure container apps #101

Open chinwobble opened 3 months ago

chinwobble commented 3 months ago

Bug Report

Symptom

A clear and concise description of what the bug is.

What is the expected behavior?

What did you expect to see? That resource labels are working.

What is the actual behavior? When deployed to azure container apps, the telemetry is getting sent to grafana but the metrics and traces don't have any resource labels. I printed out the protobuf and I am seeing this.

*
(
service.name
unknown_service:dotnet

Locally everything is working fine. I see prometheus metrics with labels, instance, job, etc

What did you see instead?

Reproduce

I have a very simple config writing to grafana cloud.

I am using these package versions

<PackageVersion Include="Grafana.OpenTelemetry.Base" Version="0.8.1-beta" />
<PackageVersion Include="OpenTelemetry.Extensions.Hosting" Version="1.8.1" />
<PackageVersion Include="OpenTelemetry.ResourceDetectors.Azure" Version="1.0.0-beta.6" />
<PackageVersion Include="OpenTelemetry.Exporter.Console" Version="1.7.0" />
var exporterSettings = configuration.CreateGrafanaOtlpExporter();
services
    .AddOpenTelemetry()
    .WithMetrics(o =>
    {
        if (metricMeters.Length > 0)
        {
            o.AddMeter(metricMeters);
        }
        o.UseGrafana(cfg => cfg.ExporterSettings = exporterSettings);
    });

Additional Context

I tried setting extra environment variables like OTEL_SERVICE_NAME. This sets the service_name but not the instance property.

chinwobble commented 3 months ago

I've tracked down the issue. When I add this line

cfg.Instrumentations.Remove(Instrumentation.AWSResource);

The issue is fixed.

pyohannes commented 3 months ago

Thanks @chinwobble for reporting this, thanks also for finding a fix!

We'll look into ways to improve this. Either we figure out a way to only activate the AWS resource when our distro is used in AWS, oherwise we'll deactivate it by default.