Open normj opened 3 months ago
Some other requirements for production-ready structured logging support:
I haven't specifically played with the new capabilities yet, but I have updated an existing lambda using .NET 8 with AoT where I use .NET's own built-in AddJsonConsole()
to log things and get this warning:
ILC : AOT analysis warning IL3050: Amazon.Lambda.RuntimeSupport.Helpers.Logging.JsonLogMessageFormatter.FormatJsonValue(Utf8JsonWriter,Object,String,MessageProperty.Directive): Using member 'System.Text.Json.JsonSerializer.Serialize<Object>(Object,JsonSerializerOptions)' which has 'RequiresDynamicCodeAttribute' can break functionality when AOT compiling. JSON serialization and deserialization might require types that cannot be statically analyzed and might need runtime code generation. Use System.Text.Json source generation for native AOT applications. [/home/runner/work/alexa-london-travel/alexa-london-travel/src/LondonTravel.Skill/LondonTravel.Skill.csproj]
This is with the provided.al2023
custom runtime. It seems that the docs are out of date though, as that runtime isn't listed as supported in the table. It is noted later that a custom runtime needs to honour the log-level settings, but I think the documentation isn't clear that it's supported (albeit not Just Works™️).
Thanks @martincostello, I'll take a look to see what we need to do.
It would be very useful to have consistent logging between lambda startup logs and .NET ILogger logs (Microsoft.Extensions.Logging). Maybe not as default since not everyone might be using it; but an option for lambdas that use Microsoft.Extensions stack.
@martincostello Sorry for the delay but I put out a PR to address the Native AOT warning. https://github.com/aws/aws-lambda-dotnet/pull/1795
Lambda JSON logging feedback request
TL;DR version
Looking for users to try the preview versions of Lambda NuGet packages supporting JSON logging. This zip file LambdaJsonLoggingPreview.zip was uploaded May 9th 2024 containing the following NuGet packages.
Intro
In November 2023, Lambda released new advanced logging controls. These new controls allow you to configure log levels and format, including JSON formatting. To take advantage of these new features, the Lambda runtime client of each language must be updated. In .NET the Lambda runtime client is the Amazon.Lambda.RuntimeSupport NuGet package with Amazon.Lambda.Core providing interfaces to access features like logging. We have been working on updating these packages to support Lambda’s advanced logging controls and are looking for feedback from the community on our implementation before rolling the changes out to NuGet and the managed runtime.
The JSON support for .NET and logging is modeled similarly to other Lambda runtimes. In our research on JSON logging, also called structured logging, points to Serilog being the most popular library of choice. Where possible this Lambda JSON support follows the patterns on Serilog. As a rule, the .NET Lambda runtime client does not take dependencies on other packages. This rule comes about for startup and maintenance reasons as well as avoiding collisions with packages brought in by the Lambda deployment bundle. This means that Amazon.Lambda.RuntimeSupport cannot directly use Serilog.
We are looking for feedback in the following areas:
Getting started
.NET Lambda functions can be written either as a class library or an executable. When deployed as an executable Amazon.Lambda.RuntimeSupport is included as part of the deployment package, as opposed to being provided by the managed runtime. Deploying an executable function allows us to try a newer version of the .NET Lambda runtime client that hasn’t been deployed to the managed runtime yet.
Project step
These steps set up a .NET Lambda project as an executable with the preview versions of the .NET Lambda runtime client using the .NET CLI.
Install the AWS .NET Lambda template into the CLI:
dotnet new install Amazon.Lambda.Templates
Create a new executable .NET Lambda project:
dotnet new lambda.EmptyTopLevelFunction -o DotNetJsonLoggingPreview
Setup local NuGet feed:
cd DotNetJsonLoggingPreview
dotnet new nugetconfig
mkdir nuget-local
dotnet nuget add source -n local ./nuget-local
./nuget-local
Update the project to use the preview .NET Lambda runtime client:
./src/DotnetJsonLoggingPreview
in your IDE of choice.PropertyGroup
element, add the following element to access preview features:<EnablePreviewFeatures>true</EnablePreviewFeatures>
Note: We have seen when having .NET 9 preview installed that we still get build errors requiring
EnablePreviewFeatures
be set even though it was added to the project. To work around this add aglobal.json
file to force the .NET 8 SDK to be used.dotnet new globaljson --sdk-version 8.0.204
Deploy project and test project
The Lambda project can be deployed with your preferred tool. This section demonstrates how to deploy using the .NET CLI tool called Amazon.Lambda.Tools, but any Lambda deployment tooling that supports .NET should work.
dotnet tool install -g amazon.lambda.tools
dotnet lambda deploy-function DotNetJsonLoggingPreview -pcfg true
(The-pcfg
flag saves the deployment configuration to theaws-lambda-tools-defaults.json
file.)To test the project from the command line, execute the following:
By default, functions will still use text based logging, so you will see an output similar to following:
The following AWS CLI command can be used to enable JSON logging. You can also enable JSON in the AWS Console. To change back to text based logging run the AWS CLI command again replacing “JSON” with “Text”
After running the previous command, invoke the function again to see that the output will be in JSON.
The new logging APIs
As part of the effort to support JSON logging, new parameterized logging methods have been added to Amazon.Lambda.Core that you downloaded in the steps above. These new APIs are the ones marked as preview, requiring the EnablePreviewFeatures property to be set. This will be required until the new version of Amazon.Lambda.RuntimeSupport has been deployed to the managed runtime for class-library-based Lambda functions. These new logging APIs will also function for the existing Text logging mode providing the string replacement.
The following code shows the handler in the project using the parameterized logging to log the input property.
The parameterized logging statement will produce the following JSON, adding the parameters to the log statement as properties of the JSON logging.
Following the pattern of other logging libraries, exceptions can also be passed in for logging. For example, the following code will throw an exception if the input is equal to “fail”.
This produces the following JSON.
Serialization options
Formatting scalers: When using scalars like integers and doubles as logging parameters, the format can be customized by providing a format string using a suffix on the parameter name. For example,
The string after the colon in {cost:0.00} is the same as the formatString used in .NET’s composite formatting. The code above produces the following JSON logging statement.
Custom types: When using instances of custom types as parameters to logging, the ToString method is used as the value for the property. If the @ prefix is used, the object will be serialized into the JSON logging statement. For example, the code below:
Produce the following JSON document:
Collections: When using lists and dictionaries as parameters, the items in the collections are written to the JSON log using the
ToString
method. The@
prefix can be used to indicate that the items should be serialized into the JSON log message.Example JSON using
ToString
:Example JSON using serialization:
Feedback
Please try these NuGet packages with enabling JSON logging. Any issues or suggestions found please add a comment to this issue. Feedback that these changes are useful to you is also helpful in letting us know if the changes are ready to be deployed.