Bug: Logger serializes bool values as true/false - Log queries with CloudWatch Insights must use 1/0

[nCubed]

Expected Behaviour

• When the Logger writes a log entry for a bool value, the log entry is true/false.
• When querying the logs with CloudWatch Insights, the bool value should be queryable with true/false

Current Behaviour

• CloudWatch Insights queries return the bool value as a 1/0
• CloudWatch Insights filters must use 1/0 for the bool value

Code snippet

Logger.LogInformation(new { @true = true, @false = false });

Cloud Insights

message.true=1 message.false=0

Possible Solution

No response

Steps to Reproduce

• Write a log entry with a boolean value.
• Retrieve/query the log with CloudWatch Insights.

Powertools for AWS Lambda (.NET) version

latest

AWS Lambda function runtime

dotnet8

Debugging logs

No response

[dreamorosi]

Hi @nCubed, thanks for opening this issue.

I've tried to reproduce the issue and I am unsure about the report.

Our Logger emits JSON structured data, and based on that boolean values are represented as true and false. These values are represented as they are in CloudWatch Logs, since they are valid JSON data types.

Once the logs are indexed in CloudWatch Log Insights, discovered fields within the structured logs appear to be stored as 1 and 0 for true and false respectively. This is not something we can control, since we only emit valid JSON objects.

It's important to note also that at no point CloudWatch Log Insights, nor CloudWatch Logs, change the data type of what we output. What you appear to be observing is only limited to how CloudWatch Log Insights stores fields, which then has implications on how you must query the data.

I tested this with another version of Powertools for AWS - which has similar outputs - and used the following query:

fields @timestamp, someVal, message, @message
| filter someVal = 1
| sort @timestamp desc
| limit 10000

As you can see from the image below, the @message (which is the reserved keyword in Log Insights to refer to the logged blob) still contains the boolean value represented as such. The discovered field however is now treated as a 1:

I could not find a mention of this behavior in the CloudWatch docs, however if I were to guess this is the result of how CloudWatch Insights stores the indexed data. Using small integers rather than boolean values might be either a cost saving measure or a necessity based on the underlying architecture of the service (which I have no knowledge of).

Back to the point at hand, I am not sure Powertools for AWS should be opinionated in regards to how boolean values are treated. I don't think indiscriminately converting them to 1s and 0s is the right choice and it would indubitably have impact on other customers.

I however understand and acknowledge that seeing boolean values in the logs, and having to use numbers to query these same logs might feel counterintuitive and inconsistent. To work around this I would suggest you to look into adopting a custom log formatter for Logger which would allow you to coerce boolean values to their corresponding numbers before each log is emitted.

[nCubed]

Thanks for the detailed reply @dreamorosi. Everything you're mentioned makes sense and what I had expected for the most part. Feel free to close out this issue report, unless there's anything else I can do to assist here.

[hjgraca]

Thanks @dreamorosi! @nCubed I will be closing the issue, thank you!