serilog / serilog-aspnetcore

Serilog integration for ASP.NET Core
Apache License 2.0
1.31k stars 205 forks source link

Serilog for Dotnet Aspire #359

Closed BartNetJS closed 8 months ago

BartNetJS commented 8 months ago

I'm exploring the new Dotnet 8 Aspire dashboard. Out of the box it is providing structured logs (from opentelemetry) and tracing as shown in the print screen: image

As soon i add Serilog Use like this

builder.Host.UseSerilog();

I loose the structured logs. For example here i added the UseSerilog in the web frontend: image

The serilog instrumentation is like this:

Log.Logger = new LoggerConfiguration() .WriteTo.Console() .Enrich.FromLogContext() .Enrich.With(new TraceIdEnricher()) .CreateLogger();

How can i use Serilog and Aspire together?

davidfowl commented 8 months ago

Did you configure Serilog to write to dashboard? (the configured OTLP server?). See this doc. It explains how the system currently works. To make serilog work with aspire, you need to tell to send serilog logs to the otlp endpoint.

https://github.com/dotnet/aspire/issues/1872

BartNetJS commented 8 months ago

indeed, i forgot to configure the OTLP exporter. I ended with this code and it started to work fine:

var useOtlpExporter = !string.IsNullOrWhiteSpace(builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]);
var logBuilder = new LoggerConfiguration()
 .Enrich.FromLogContext()
 .WriteTo.Console();

if (useOtlpExporter)
{
    logBuilder
       .WriteTo.OpenTelemetry(options =>
     {
         options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"];
         options.ResourceAttributes.Add("service.name", "apiservice");
     });
}

Log.Logger = logBuilder.CreateBootstrapLogger();

builder.Logging.AddSerilog();
davidfowl commented 8 months ago

I filed https://github.com/dotnet/aspire-samples/issues/106

Sen-Gupta commented 8 months ago

Hello @BartNetJS

We attempted adding Serilog as well. I Need a confirmation! Are you seeing two entries? As shown below,

@davidfowl is this expected?

image

Created another static method in ServiceDefaults

public static void AddCustomSerilog(this IHostApplicationBuilder builder) { var useOtlpExporter = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]; var appName = builder.Configuration["AppMeta:AppName"];

if (string.IsNullOrEmpty(appName))
{
    throw new ArgumentException("Must set AppMeta.AppName for serilog to work correctly in the configuration");
}

var seqServerUrl = builder.Configuration["SeqServerUrl"]!;

var logBuilder = new LoggerConfiguration()
    .MinimumLevel.Debug()
    .ReadFrom.Configuration(builder.Configuration)
    .Enrich.FromLogContext()
    .WriteTo.Console()
    .WriteTo.Seq(seqServerUrl)
    .Enrich.FromLogContext()
    .Enrich.WithMachineName()
    .Enrich.WithExceptionDetails();

if(!string.IsNullOrEmpty(useOtlpExporter))
{
    logBuilder
       .WriteTo.OpenTelemetry(options =>
       {
           options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]!;
           options.ResourceAttributes.Add("service.name", appName);
       });
}
Log.Logger = logBuilder.CreateBootstrapLogger();
builder.Logging.AddSerilog();

}

Note: Ignore the Seq bits!

The App Name is configured in settings as shown below "AppMeta": { "AppMode": "Server", "Env": "qa", "StateStore": "aarya-statestore", "AppName": "agents", "ProductName": "Insights" }

and finally the projects are added in the AppHost as shown below:

var builder = DistributedApplication.CreateBuilder(args);

builder.AddProject("agentsgrpc") .WithDaprSidecar("agents");

builder.AddProject("analyticsgrpc") .WithDaprSidecar("analytics");

builder.AddProject("collectorgrpc") .WithDaprSidecar("collector");

builder.Build().Run();

as we see AppName:"agents" and sidecare is registered with "agents".

But seems, telemetry is identifying them as two different sources and appending random strings to ensure listing of the sources.

Any clues?

davidfowl commented 8 months ago

Did you remove the other logger provider that sends telemetry directly to the dashboard or are you logging with both serilog and the default setup?

Sen-Gupta commented 8 months ago

@davidfowl @TGrannen,

The only extra thing needed was to clear the providers, it was already there in our production code but I missed during customiztation of the code snippet.

public static void AddCustomSerilog(this IHostApplicationBuilder builder) { var useOtlpExporter = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]; var appName = builder.Configuration["AppMeta:AppName"];

if (string.IsNullOrEmpty(appName)) { throw new ArgumentException("Must set AppMeta.AppName for serilog to work correctly in the configuration"); }

var seqServerUrl = builder.Configuration["SeqServerUrl"]!;

var logBuilder = new LoggerConfiguration() .MinimumLevel.Debug() .ReadFrom.Configuration(builder.Configuration) .Enrich.FromLogContext() .WriteTo.Console() .WriteTo.Seq(seqServerUrl) .Enrich.FromLogContext() .Enrich.WithMachineName() .Enrich.WithExceptionDetails();

if(!string.IsNullOrEmpty(useOtlpExporter)) { logBuilder .WriteTo.OpenTelemetry(options => { options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]!; options.ResourceAttributes.Add("service.name", appName); }); } Log.Logger = logBuilder.CreateBootstrapLogger(); builder.Logging.ClearProviders().AddSerilog(); }

Seems like @TGrannen has beaten me for the sample. ;-)

alex-todorov-j2 commented 5 months ago

This used to work - until I updated to latest VS Preview. Now only the Console and File sinks create Serilog logs but I get nothing in Structured in Dashboard. I confirmed that OTEL_EXPORTER_OTLP_ENDPOINT is correct. And if I call ClearProviders before AddSerilog, I get no structured logs in any sink. Any ideas?

davidfowl commented 5 months ago

https://learn.microsoft.com/en-us/dotnet/aspire/whats-new/preview-5?tabs=dotnet-cli#security-updates

alex-todorov-j2 commented 5 months ago

I added in appsettings.json:

"Dashboard": {
  "Otlp": {
    "AuthMode": "ApiKey",
    "ApiKey": "a8e18bea-609d-4761-aa6b-40858d7c057b"
  }    
}

And now the code sends the configured api key to the OTLP collector endpoint in the extension method:

public static IHostApplicationBuilder AddOtelSerilog(this IHostApplicationBuilder builder, IConfiguration config)
{
    ...
    logBuilder.WriteTo.OpenTelemetry(options =>
    {
        options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]; 
        options.Headers.Add("x-otlp-api-key", otlpApiKey);
        options.ResourceAttributes.Add("service.name", "apiservice");                
    });
    ....
}

But still I don't see the logs in Structured in Dashboard.

What else has to change?

alex-todorov-j2 commented 5 months ago

Any update? Should I open an issue for this?

davidfowl commented 5 months ago

Easiest way to make this would is to read the official environment variables and passing them to serilog (https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/#otel_exporter_otlp_headers)

OTEL_EXPORTER_OTLP_ENDPOINT OTEL_EXPORTER_OTLP_HEADERS OTEL_EXPORTER_OTLP_PROTOCOL

logBuilder.WriteTo.OpenTelemetry(options =>
{
    options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]; 
    var headers = builder.Configuration["OTEL_EXPORTER_OTLP_HEADERS"]?.Split(',') ?? [];
    foreach (var header in headers)
    {
        var (key, value) = header.Split('=') switch
        {
            [string k, string v] => (k, v),
            var v => throw new Exception($"Invalid header format {v}")
        };

        options.Headers.Add(key, value);
    }
    options.ResourceAttributes.Add("service.name", "apiservice");                
});

PS: It would be good to file an issue on serilg to read the default OTLP environment variables.

alex-todorov-j2 commented 5 months ago

This works, thanks! The only thing is that it creates a second "apiservice" log in Structured. Even with builder.Logging.ClearProviders().AddSerilog(); Any idea why? I have commented out the Logging section in appsettings.

I'll create an issue for the headers.

davidfowl commented 5 months ago

Even with builder.Logging.ClearProviders().AddSerilog(); Any idea why?

Where are you clearing providers? Before or after the otlp logger?

alex-todorov-j2 commented 5 months ago

Tried both before and after, but neither remove the duplicate log:

public static IHostApplicationBuilder AddOtelSerilog(this IHostApplicationBuilder builder, IConfiguration config)
{
    var serilogConfig = new ConfigurationReaderOptions() { SectionName = "Serilog" };
    LoggerConfiguration logBuilder = new LoggerConfiguration().ReadFrom.Configuration(builder.Configuration, serilogConfig);        

    logBuilder.WriteTo.OpenTelemetry(options =>
    {
        options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"];
        var headers = builder.Configuration["OTEL_EXPORTER_OTLP_HEADERS"]?.Split(',') ?? [];

        foreach (var header in headers)
        {
            var (key, value) = header.Split('=') switch
            {
                [string k, string v] => (k, v),
                _ => throw new Exception($"Invalid header format {header}")
            };

            options.Headers.Add(key, value);
        }

        options.ResourceAttributes.Add("service.name", "apiservice");                
    });

    Log.Logger = logBuilder.CreateBootstrapLogger();
    //builder.Logging.ClearProviders(); //before
    builder.Logging.AddSerilog();           
    //builder.Logging.ClearProviders(); //after

    return builder;
}
davidfowl commented 5 months ago

Where is this called in relation to AddServiceDefaults?

alex-todorov-j2 commented 5 months ago

After builder.AddServiceDefaults();

alex-todorov-j2 commented 5 months ago

I used the actual service name to register the Serilog log resource for the apiservice, e.g. options.ResourceAttributes.Add("service.name", "NetAspireStarter1.ApiService"); This solves the duplication problem and you simply filter by that name to see its Serilog logs.

chinmay-trivedi commented 5 months ago

Easiest way to make this would is to read the official environment variables and passing them to serilog (https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/#otel_exporter_otlp_headers)

OTEL_EXPORTER_OTLP_ENDPOINT OTEL_EXPORTER_OTLP_HEADERS OTEL_EXPORTER_OTLP_PROTOCOL

logBuilder.WriteTo.OpenTelemetry(options =>
{
    options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]; 
    var headers = builder.Configuration["OTEL_EXPORTER_OTLP_HEADERS"]?.Split(',') ?? [];
    foreach (var header in headers)
    {
        var (key, value) = header.Split('=') switch
        {
            [string k, string v] => (k, v),
            var v => throw new Exception($"Invalid header format {v}")
        };

        options.Headers.Add(key, value);
    }
    options.ResourceAttributes.Add("service.name", "apiservice");                
});

PS: It would be good to file an issue on serilg to read the default OTLP environment variables.

@davidfowl Thanks for your reply. This is really helpful.

islamkhattab commented 5 months ago

I had the same duplication issue and it worked for me when I added the the OTEL_RESOURCE_ATTRIBUTES to Serilog options

var (otelResourceAttribute, otelResourceAttributeValue) = configuration["OTEL_RESOURCE_ATTRIBUTES"]?.Split('=') switch
{
   [string k, string v] => (k, v),
    _ => throw new Exception($"Invalid header format {configuration["OTEL_RESOURCE_ATTRIBUTES"]}")
};

options.ResourceAttributes.Add(otelResourceAttribute, otelResourceAttributeValue);
Blind-Striker commented 4 months ago

I've been facing a similar issue with using Serilog alongside OpenTelemetry, particularly around logs either not being sent or being duplicated. After fixing the configuration using suggestions from @davidfowl, logs started showing up correctly but were duplicated. Following @islamkhattab advice, logs appeared under the same service, but duplication persisted. Both logs from OpenTelemetry and Serilog were appearing under the same service. For some reason, even after removing the OpenTelemetry logging provider, logs were still being sent from there.

Using Logging.AddSerilog doesn't prevent duplication, regardless of where ClearProviders is used.

The solution I found was to use Serilog's UseSerilog method in the IHostBuilder extension. However, the Host property is only exposed for WebApplicationBuilder, so if you're using a console application and HostApplicationBuilder, this problem remains unresolved. After some deep diving, I discovered that HostApplicationBuilder has an internal method called AsHostBuilder. By calling this method using reflection and then calling UseSerilog, I managed to resolve this issue for console apps as well.

When Serilog is registered using the IHostBuilder extension, it somehow ensures that it replaces other logging providers and remains the sole logging provider. I will dive into Serilog's code to understand this better. For now, I've come up with a temporary solution as shown below.

public static IHostApplicationBuilder ConfigureSerilog(this IHostApplicationBuilder builder)
{
    ArgumentNullException.ThrowIfNull(builder);

    builder.Logging.ClearProviders();

    Action<LoggerConfiguration> configureLogger = config =>
    {
        config.ReadFrom.Configuration(builder.Configuration)
              .Enrich.FromLogContext()
              .Enrich.WithMachineName()
              .Enrich.WithProcessId()
              .Enrich.WithProcessName()
              .Enrich.WithThreadId()
              .Enrich.WithSpan()
              .Enrich.WithExceptionDetails(new DestructuringOptionsBuilder()
                                           .WithDefaultDestructurers()
                                           .WithDestructurers(new[] { new DbUpdateExceptionDestructurer() }))
              .Enrich.WithProperty("Environment", builder.Environment.EnvironmentName)
              .WriteTo.Console()
              .WriteTo.OpenTelemetry(options =>
              {
                  options.IncludedData = IncludedData.TraceIdField | IncludedData.SpanIdField;
                  options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"];
                  AddHeaders(options.Headers, builder.Configuration["OTEL_EXPORTER_OTLP_HEADERS"]);
                  AddResourceAttributes(options.ResourceAttributes, builder.Configuration["OTEL_RESOURCE_ATTRIBUTES"]);

                  void AddHeaders(IDictionary<string, string> headers, string headerConfig)
                  {
                      if (!string.IsNullOrEmpty(headerConfig))
                      {
                          foreach (var header in headerConfig.Split(','))
                          {
                              var parts = header.Split('=');

                              if (parts.Length == 2)
                              {
                                  headers[parts[0]] = parts[1];
                              }
                              else
                              {
                                  throw new Exception($"Invalid header format: {header}");
                              }
                          }
                      }
                  }

                  void AddResourceAttributes(IDictionary<string, object> attributes, string attributeConfig)
                  {
                      if (!string.IsNullOrEmpty(attributeConfig))
                      {
                          var parts = attributeConfig.Split('=');

                          if (parts.Length == 2)
                          {
                              attributes[parts[0]] = parts[1];
                          }
                          else
                          {
                              throw new Exception($"Invalid resource attribute format: {attributeConfig}");
                          }
                      }
                  }
              });
    };

    switch (builder)
    {
        case WebApplicationBuilder webApplicationBuilder:
            webApplicationBuilder.Host.UseSerilog((_, _, options) => configureLogger(options));
            break;
        case HostApplicationBuilder hostApplicationBuilder when
            hostApplicationBuilder.GetType().GetMethod("AsHostBuilder", BindingFlags.Instance | BindingFlags.NonPublic) is { } asHostBuilderMethod:
        {
            if (asHostBuilderMethod.Invoke(hostApplicationBuilder, parameters: null) is not IHostBuilder hostBuilder)
            {
                throw new InvalidOperationException("Failed to get IHostBuilder from HostApplicationBuilder");
            }

            hostBuilder.UseSerilog((_, _, options) => configureLogger(options));
            break;
        }
        default:
        {
            var loggerConfig = new LoggerConfiguration();
            configureLogger(loggerConfig);
            builder.Logging.AddSerilog(loggerConfig.CreateLogger());

            break;
        }
    }

    return builder;
}
nblumhardt commented 4 months ago

@Blind-Striker FWIW the builder.Services.AddSerilog() method works with all of these hosting models and achieves the same result (replacing the default ILoggerFactory with the Serilog-provided one). HTH!

Blind-Striker commented 4 months ago

Ok, it seems that Services.AddSerilog solves the issue for all hosting models. Since I'm new to the MinimalApi world, I hadn't noticed this before. Until now, I had always used the UseSerilog method from the IHostBuilder extension. It appears that Logging.AddSerilog and Services.AddSerilog behave differently. As their documentation states:

(P.S. to myself: read the fricking manual next time 🙃)

I'm still not sure why ClearProviders doesn't remove all other providers or how logs from OpenTelemetry continue to be sent alongside Serilog, causing duplicate logging, even after completely removing the OpenTelemetry logging provider. But the final code is shown below:

public static IHostApplicationBuilder ConfigureSerilog(this IHostApplicationBuilder builder)
{
    ArgumentNullException.ThrowIfNull(builder);

    builder.Services.AddSerilog(config =>
    {
        config.ReadFrom.Configuration(builder.Configuration)
              .Enrich.FromLogContext()
              .Enrich.WithMachineName()
              .Enrich.WithProcessId()
              .Enrich.WithProcessName()
              .Enrich.WithThreadId()
              .Enrich.WithSpan()
              .Enrich.WithExceptionDetails(new DestructuringOptionsBuilder()
                                           .WithDefaultDestructurers()
                                           .WithDestructurers(new[] { new DbUpdateExceptionDestructurer() }))
              .Enrich.WithProperty("Environment", builder.Environment.EnvironmentName)
              .WriteTo.Console()
              .WriteTo.OpenTelemetry(options =>
              {
                  options.IncludedData = IncludedData.TraceIdField | IncludedData.SpanIdField;
                  options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"];
                  AddHeaders(options.Headers, builder.Configuration["OTEL_EXPORTER_OTLP_HEADERS"]);
                  AddResourceAttributes(options.ResourceAttributes, builder.Configuration["OTEL_RESOURCE_ATTRIBUTES"]);

                  void AddHeaders(IDictionary<string, string> headers, string headerConfig)
                  {
                      if (!string.IsNullOrEmpty(headerConfig))
                      {
                          foreach (var header in headerConfig.Split(','))
                          {
                              var parts = header.Split('=');

                              if (parts.Length == 2)
                              {
                                  headers[parts[0]] = parts[1];
                              }
                              else
                              {
                                  throw new Exception($"Invalid header format: {header}");
                              }
                          }
                      }
                  }

                  void AddResourceAttributes(IDictionary<string, object> attributes, string attributeConfig)
                  {
                      if (!string.IsNullOrEmpty(attributeConfig))
                      {
                          var parts = attributeConfig.Split('=');

                          if (parts.Length == 2)
                          {
                              attributes[parts[0]] = parts[1];
                          }
                          else
                          {
                              throw new Exception($"Invalid resource attribute format: {attributeConfig}");
                          }
                      }
                  }
              });
    });

    return builder;
}
Blind-Striker commented 4 months ago

@Blind-Striker FWIW the builder.Services.AddSerilog() method works with all of these hosting models and achieves the same result (replacing the default ILoggerFactory with the Serilog-provided one). HTH!

@nblumhardt, sorry, I had reached the same conclusion, just before saw your comment. Thank you very much! 🙏

softmatters commented 4 months ago

@nblumhardt Thanks for the clarification around builder.Services.AddSerilog(). I was also using UseSerilog.

@Blind-Striker Thanks a lot for providing a fix. I had the exact same problem. It has fixed the issue with structured logs, and I can also see the traces without duplicate names. However, with this fix, I am now seeing an Unknown entry in Structured logs and Traces for my apiService. Please see blow. It's showing correctly under Resources and Console tabs. Any thoughts?

image

image

image

My setup is like below. Please note that my apiService is in a different repository, and not part of the solution, so I am loading it via ProjectPath.

AppHost: Program.cs


using Microsoft.Extensions.Configuration;
using Portal.AppHost.Configuration;

var builder = DistributedApplication.CreateBuilder(args);

var configuration = builder.Configuration;

// get the configuration settings for the projects
var projectSettingsSection = configuration.GetSection(nameof(ProjectsSettings));
var projectSettings = projectSettingsSection.Get<ProjectsSettings>();

// get the configuration settings for api service
var apiServiceSettings = projectSettings.ApiServiceSettings;

// add the project reference
var apiService = builder.AddProject(apiServiceSettings.ProjectName, apiServiceSettings.ProjectPath);

// add the portal project with a dependency on
// apiService
builder
    .AddProject<Projects._Portal>("xxx-xxx-portal")
    .WithReference(apiService);

// run the host app
await builder.Build().RunAsync();

ServiceDefaults: Extensions.cs


This is exactly the same as yours apart from my Serilog setup is a bit trimmed down as I am reading from file.

public static IHostApplicationBuilder ConfigureSerilog(this IHostApplicationBuilder builder)
{
    ArgumentNullException.ThrowIfNull(builder);

    builder.Services.AddSerilog(config =>
    {
        config
            .ReadFrom.Configuration(builder.Configuration)
            .WriteTo.OpenTelemetry(options =>
            {
                options.IncludedData = IncludedData.TraceIdField | IncludedData.SpanIdField;
                options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"];
                AddHeaders(options.Headers, builder.Configuration["OTEL_EXPORTER_OTLP_HEADERS"]);
                AddResourceAttributes(options.ResourceAttributes, builder.Configuration["OTEL_RESOURCE_ATTRIBUTES"]);

                void AddHeaders(IDictionary<string, string> headers, string headerConfig)
                {
                    if (!string.IsNullOrEmpty(headerConfig))
                    {
                        foreach (var header in headerConfig.Split(','))
                        {
                            var parts = header.Split('=');
                            if (parts.Length == 2)
                            {
                                headers[parts[0]] = parts[1];
                            }
                            else
                            {
                                throw new InvalidOperationException($"Invalid header format: {header}");
                            }
                        }
                    }
                }

                void AddResourceAttributes(IDictionary<string, object> attributes, string attributeConfig)
                {
                    if (!string.IsNullOrEmpty(attributeConfig))
                    {
                        var parts = attributeConfig.Split('=');
                        if (parts.Length == 2)
                        {
                            attributes[parts[0]] = parts[1];
                        }
                        else
                        {
                            throw new InvalidOperationException($"Invalid resource attribute format: {attributeConfig}");
                        }
                    }
                }
            });
    });

    return builder;
}
leojus commented 4 months ago

Hello everyone! I have successfully set up the Aspire dashboard as a standalone dashboard to one of my web apis. I'm able to read metrics, logs and traces in the dashboard without problem. I'm spinning up the dashboard as a docker compose locally, but everytime I do so im getting this message in my dashboard:

Skärmbild 2024-05-23 090553

I have tried several methods, including the one @davidfowl mentioned with passing in them in the launchSettings. I have provided different variables to my docker compose - but still getting the message. Can someone point me in the right direction? Or does anyone have a solution?

davidfowl commented 4 months ago

Did you click the more information link? It points to docs that explain how to set it up securely

Blind-Striker commented 4 months ago

@nblumhardt Thanks for the clarification around builder.Services.AddSerilog(). I was also using UseSerilog.

@Blind-Striker Thanks a lot for providing a fix. I had the exact same problem. It has fixed the issue with structured logs, and I can also see the traces without duplicate names. However, with this fix, I am now seeing an Unknown entry in Structured logs and Traces for my apiService. Please see blow. It's showing correctly under Resources and Console tabs. Any thoughts?

Hey @softmatters, I'm glad you managed to resolve the duplicate message issue. I'll take a closer look and try to respond in more detail. In the meantime, I released a demo using Aspire, OpenTelemetry, and Serilog yesterday. If you review the code and compare it with yours, you might fix the problem faster.

https://github.com/Blind-Striker/dotnet-otel-aspire-localstack-demo

softmatters commented 4 months ago

Hey @softmatters, I'm glad you managed to resolve the duplicate message issue. I'll take a closer look and try to respond in more detail. In the meantime, I released a demo using Aspire, OpenTelemetry, and Serilog yesterday. If you review the code and compare it with yours, you might fix the problem faster.

https://github.com/Blind-Striker/dotnet-otel-aspire-localstack-demo

Hi @Blind-Striker , thanks a lot. I'll take a look and will let you know if I managed to resolve the issue or not.

leojus commented 4 months ago

Did you click the more information link? It points to docs that explain how to set it up securely

Hehe... I had a typo in my api key :) But thanks!

AlbertoMonteiro commented 3 months ago

I do like to use Serilog configuration from appSettings.

I changed the code shared here a little bit to make it work with that approach.

So if you have given appSettings.Development.json

"Serilog": {
  "Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.OpenTelemetry" ],
  "MinimumLevel": "Debug",
  "WriteTo": [
    {
      "Name": "Console",
      "Args": {
        "outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3} {CorrelationId}] {Message:lj}{NewLine}{Exception}"
      }
    },
    {
      "Name": "OpenTelemetry",
      "Args": {
        "endpoint": "%OTEL_EXPORTER_OTLP_ENDPOINT%",
        "includedData": "TraceIdField, SpanIdField"
      }
    }
  ],
  "Enrich": [
    {
      "Name": "WithProperty",
      "Args": {
        "name": "ApplicationName",
        "value": "PocRedisOtel.Api"
      }
    },
    "WithMachineName",
    "WithDemystifiedStackTraces",
    "WithClientAgent",
    "FromLogContext",
    "WithCorrelationIdHeader"
  ]
}

I've created the method AddSerilogOtelSection that set those values in the Configuration.

void AddSerilogOtelSection(string config, string section)
{
    foreach (var part in config?.Split(',') ?? [])
    {
        if (part.Split('=') is [var key, var value])
            builder.Configuration[$"Serilog:WriteTo:1:Args:{section}:{key}"] = value;
        else
            throw new InvalidOperationException($"Invalid {section} format: {part}");
    }
}

Then I just call it

var builder = WebApplication.CreateBuilder(args);

AddSerilogOtelSection(builder.Configuration["OTEL_EXPORTER_OTLP_HEADERS"], "headers");
AddSerilogOtelSection(builder.Configuration["OTEL_RESOURCE_ATTRIBUTES"], "resourceAttributes");
davidfowl commented 3 months ago

Did anyone file an issue to make sure the serilog otel extensions natively support these configuration options?

AlbertoMonteiro commented 3 months ago

I guess so @davidfowl

https://github.com/serilog/serilog-sinks-opentelemetry/issues/12

AlbertoMonteiro commented 3 months ago

The version 4.0.0-dev-00313 of the Serilog.Sinks.OpenTelemetry will be able to read the environment variables by default

https://www.nuget.org/packages/Serilog.Sinks.OpenTelemetry/4.0.0-dev-00313

image

"Serilog": {
  "Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.OpenTelemetry" ],
  "MinimumLevel": "Debug",
  "WriteTo": [
    {
      "Name": "Console",
      "Args": {
        "outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3} {CorrelationId}] {Message:lj}{NewLine}{Exception}"
      }
    },
    {
      "Name": "OpenTelemetry"
    }
  ],
  //....
}

Thank you @nblumhardt for helping me!!!

davidfowl commented 3 months ago

Great contribution @AlbertoMonteiro !

sbalocky commented 2 months ago

The version 4.0.0-dev-00313 of the Serilog.Sinks.OpenTelemetry will be able to read the environment variables by default

https://www.nuget.org/packages/Serilog.Sinks.OpenTelemetry/4.0.0-dev-00313

image

"Serilog": {
  "Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.OpenTelemetry" ],
  "MinimumLevel": "Debug",
  "WriteTo": [
    {
      "Name": "Console",
      "Args": {
        "outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3} {CorrelationId}] {Message:lj}{NewLine}{Exception}"
      }
    },
    {
      "Name": "OpenTelemetry"
    }
  ],
  //....
}

Thank you @nblumhardt for helping me!!!

@AlbertoMonteiro It works, however as you can see on the picture it adds the 'unknown_service' prefix to every log item.

nblumhardt commented 2 months ago

@sbalocky https://github.com/serilog/serilog-sinks-opentelemetry/pull/146 will resolve this; HTH

gmij commented 2 months ago

这里面有个问题, 微软默认的日志记录器, 是可以实现,把日志输出到aspire,然后由aspire再转发到seq进行存储的,而应用的日志,只需要输出到console, 为什么serilog不使用相同的方式,而使用了open-telemetry这种方式呢?

https://learn.microsoft.com/zh-cn/dotnet/aspire/logging/seq-component?tabs=dotnet-cli

gmij commented 2 months ago

There is a problem here. Microsoft's default logger can output logs to Aspire, which then forwards them to Seq for storage. However, application logs only need to be output to the console. Why does Serilog use the same method instead of Open Strategy?

levinhtxbt commented 1 month ago

I still have duplicate message logs and traces when I use both OpenTelemetry and Seq. Has anyone encountered the same issue?

nblumhardt commented 1 month ago

Hi folks! If you have an additional question or feature request, a new thread on Stack Overflow (usage questions) or a new ticket in the appropriate repo (feature requests) will get the right eyes on it and improve chances of finding a successful resolution. Thanks!