Amazon.Lambda.Logging.AspNetCore should support structural json logging

[Dreamescaper]

Describe the feature

I should be able to use structural logging with lambda logger.

Use Case

I'd like to use structural logging either with CloudWatch Insights, or with third party system (e.g. DataDog).

Proposed Solution

Add Json lambda logger to DI via smth like this:

serviceCollection.AddLogging(logging =>
        {
            var loggerOptions = new LambdaJsonLoggerOptions
            {
                 ...
            };

            // Configure Lambda logging
            logging.AddLambdaJsonLogger(loggerOptions);
        });

This logger should support logging "state" object: logger.LogInformation("Saving entity {entityId} by user {userId}.", entityId, userId);

"message": "Saving entity 123456ab-29c5-4912-aa3b-65bf845ae68b for user 1234567890.",
"state": {
    "{OriginalFormat}": "Saving transaction {transactionId} for tenant {userId}.",
    "entityId": "123456ab-29c5-4912-aa3b-65bf845ae68b",
    "userId": "1234567890"
},

This logger should support logging "scopes" object:

        using var scope = logger.BeginScope(new Dictionary<string, object>
        {
            ["requestId"] = input.RequestContext?.RequestId,
            ["userId"] = "testUser"
        });
        logger.LogInformation("Info");

"message": "Info",
"scopes": {
    "requestId": "QnU_ABCDFiAEJ9Q=",
    "userId": "testUser"
},

Ideally, if I use json logger, it shouldn't be needed to set 'AWS_LAMBDA_HANDLER_LOG_FORMAT' environment variable.

Other Information

No response

Acknowledgements

• [X] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

AWS .NET SDK and/or Package version used

Amazon.Lambda.Logging.AspNetCore Version="3.1.0"

Targeted .NET Platform

.NET6

Operating System and version

AmazonLinux

[ashishdhingra]

@Dreamescaper Thanks for submitting feature request. Please advise on how you came up with the JSON format of the log output. Did you use Serilog structured logging as reference?

[Dreamescaper]

@ashishdhingra No, mostly on MS json console logger.

https://docs.microsoft.com/ru-ru/dotnet/core/extensions/console-log-formatter#json

[petarrepac]

@Dreamescaper can you setup your logging using MS Json logging and direct it to the console (std out) ?

that way you can use whatever MS supports including structured logging and not have any dependency on AWS .NET assemblies

[Dreamescaper]

@petarrepac MS console logging uses a background thread for logging, and it's not a great idea for lambdas, according to this info https://github.com/aws/aws-logging-dotnet#aws-lambda

[ashishdhingra]

Needs review with the team.

[ashishdhingra]

@Dreamescaper Good afternoon. Structured logging support was added as part of PR https://github.com/aws/aws-lambda-dotnet/pull/1803. Please refer to https://github.com/aws/aws-lambda-dotnet/issues/1747 for details. Kindly confirm if it covers your scenario.

Thanks, Ashish

[Dreamescaper]

@ashishdhingra

Not sure. Per my understanding, in order to use it with AspNetCore, it would have to implement Microsoft.Extensions.Logging.ILogger, right?