serilog-mssql / serilog-sinks-mssqlserver

A Serilog sink that writes events to Microsoft SQL Server and Azure SQL
Apache License 2.0
276 stars 147 forks source link

Custom Columns where the Property Name contains a dot is not populated. #542

Open tjackadams opened 2 months ago

tjackadams commented 2 months ago

Please clearly describe what the SQL Sink is doing incorrectly:

custom columns where the property name contains a dot, but isn't hierarchical doesn't seem to get populated.

Please clearly describe the expected behavior:

Even though it contains a dot, it should be able to match on the property name.

List the names and versions of all Serilog packages used in the project:

<PackageReference Include="Serilog.AspNetCore" Version="8.0.1" />
<PackageReference Include="Serilog.Enrichers.Environment" Version="3.0.0" />
<PackageReference Include="Serilog.Expressions" Version="4.0.0" />
<PackageReference Include="Serilog.Formatting.Compact" Version="3.0.0" />
<PackageReference Include="Serilog.Sinks.Async" Version="2.0.0" />
<PackageReference Include="Serilog.Sinks.MSSqlServer" Version="6.6.1" />

Target framework and operating system:

.NET 8 Windows 11

Provide a simple reproduction of your Serilog configuration code:

    builder.Host.UseSerilog((hostContext, loggerBuilder) =>
    {
        loggerBuilder
        .ReadFrom.Configuration(hostContext.Configuration);
    });

Provide a simple reproduction of your Serilog configuration file, if any:

{
  "Serilog": {
    "Using": [ "Serilog.Sinks.MSSqlServer" ],
    "WriteTo": [
      {
        "Name": "Logger",
        "Args": {
          "configureLogger": {
            "WriteTo": [
              {
                "Name": "MSSqlServer",
                "Args": {
                  "columnOptionsSection": {
                    "addStandardColumns": [ "LogEvent", "TraceId", "SpanId" ],
                    "removeStandardColumns": [ "MessageTemplate", "Properties" ],
                    "customColumns": [
                      {
                        "columnName": "RequestId",
                        "dataType": "varchar",
                        "dataLength": 128
                      },
                      {
                        "columnName": "Method",
                        "PropertyName": "http.request.method",
                        "dataType": "nvarchar"
                      },
                      {
                        "columnName": "Path",
                        "PropertyName": "url.path",
                        "dataType": "nvarchar"
                      },
                      {
                        "columnName": "StatusCode",
                        "PropertyName": "http.response.status_code",
                        "dataType": "int"
                      },
                      {
                        "columnName": "RequestBody",
                        "dataType": "nvarchar"
                      },
                      {
                        "columnName": "ResponseBody",
                        "dataType": "nvarchar"
                      }
                    ],
                    "id": {
                      "columnName": "LogId",
                      "dataType": "bigint",
                      "nonClusteredIndex": false
                    },
                    "level": {
                      "columnName": "Level",
                      "storeAsEnum": false
                    },
                    "timeStamp": {
                      "columnName": "Timestamp",
                      "convertToUtc": false
                    },
                    "logEvent": {
                      "columnName": "LogEvent"
                    }
                  },
                  "connectionString": "Local",
                  "sinkOptionsSection": {
                    "tableName": "HttpClientLog",
                    "autoCreateSqlDatabase": true,
                    "autoCreateSqlTable":  true
                  },
                  "formatProvider": "System.Globalization.CultureInfo::CurrentCulture",
                  "restrictedToMinimumLevel": "Information"
                }
              }
            ],
            "Filter": [
              {
                "Name": "ByIncludingOnly",
                "Args": {
                  "expression": "SourceContext = 'Microsoft.Extensions.Http.Logging.HttpClientLogger'"
                }
              }
            ]
          }
        }
      }
    ]
  }
}

Provide a simple reproduction of your application code:

https://github.com/tjackadams/serilog-mssql-dot-property-name-repro

It uses the local db that comes with visual studio. When you run it, call posts/5 via swagger and it logs the request in the database. The Method, Path and StatusCode columns will all be null, even though they are supplied as a property. The properties are populated by the Microsoft.Extensions.Http.Diagnostics nuget package and AddExtendedHttpClientLogging call in program.

This is a LogEvent from that call

{
   "Timestamp":"2024-06-11T15:40:36.8575483",
   "Level":"Information",
   "Message":"GET jsonplaceholder.typicode.com//posts/5",
   "Properties":{
      "LatencyInfo":",",
      "http.request.method":"GET",
      "server.address":"jsonplaceholder.typicode.com",
      "url.path":"/posts/5",
      "Duration":519,
      "http.response.status_code":200,
      "RequestBody":"",
      "ResponseBody":"{\n  \"userId\": 1,\n  \"id\": 5,\n  \"title\": \"nesciunt quas odio\",\n  \"body\": \"repudiandae veniam quaerat sunt sed\\nalias aut fugiat sit autem sed est\\nvoluptatem omnis possimus esse voluptatibus quis\\nest aut tenetur dolor neque\"\n}",
      "LoggerMessageState":"GET jsonplaceholder.typicode.com//posts/5",
      "EventId":{
         "Id":1,
         "Name":"OutgoingRequest"
      },
      "SourceContext":"Microsoft.Extensions.Http.Logging.HttpClientLogger",
      "RequestId":"0HN4A4HNQNRGV:00000001",
      "RequestPath":"/posts/5",
      "ConnectionId":"0HN4A4HNQNRGV",
      "MachineName":"DESKTOP-KAHG7M1"
   },
   "Renderings":{
      "LoggerMessageState":[
         {
            "Format":"l",
            "Rendering":"GET jsonplaceholder.typicode.com//posts/5"
         }
      ]
   }
}

The http.request.method value does not populate the Method column - same with url.path and http.response.status_code. However, other properties such as RequestBody and ResponseBody which also come from the Microsoft.Extensions.Http.Diagnostics nuget package are populated ok.