fluent-plugins-nursery / fluent-plugin-cloudwatch-logs

CloudWatch Logs Plugin for Fluentd
MIT License
201 stars 141 forks source link

Nested JSON output #256

Open atze234 opened 9 months ago

atze234 commented 9 months ago

Problem

Hello,

i'm parsing my logs using Grok Patterns with the fluent-plugin-grok-parser. Is it possible somehow that i can put nested JSON directly to cloudwatch?

...

Steps to replicate

Having such a pattern in filter for example:

<filter matcher>
  @type parser
  key_name message
  reserve_data true
<grok>
  pattern "^%{IP:[http][request][client][ip]} %{DATA}"
</grok>
</filter>

This will create the Field "[http][request][client][ip]" in cloudwatch logs.

Expected Behavior or What you need to ask

What i expect in Cloudwatch Logs:

{

    "host": "myhostname",
    "message": "192.168.0.1 message-from-ip",
    "http": {
        "request": {
          "client": {
            "ip" : "192.168.0.1"
          }
        }
      }
}

But i got in cloudwatch:

{

    "host": "myhostname",
    "message": "192.168.0.1 message-from-ip",
    "[http][request][client][ip]": "192.168.0.1"
}

Using Fluentd and CloudWatchLogs plugin versions