AWS Cloudwatch logs are not generally "discoverable"

[cch1]

Based on my experience, the cloudwatch publisher generates JSON logs that are not discoverable when using namespaced keys. This is a serious impediment to to idiomatic Clojure usage and the usefulness of the logs.

I believe the problem stems from the JSON serialization of the keys of logged attributes. A key like :my.qualified/key would be idiomatic in Clojure, but the serialized JSON object appears to be my.qualified/key which is apparently not discovered by Cloudwatch Log insights. Furthermore, the key names are so foreign to Cloudwatch logs that even specifying them (e.g. in a filter) is not possible without special quote-escaping syntax.

Cloudwatch logs should adhere to the "dot notation" convention outlined in this spec in order to facilitate insights, queries and discoverability. I think a reasonable solution would be to aggregate keys of a namespace into a subordinate JSON object:

{"my": {"qualified": {"key": 123}}}

If this approach is deemed too radical, perhaps allowing for a customization of the publisher would be appropriate.

[BrunoBonacci]

Hi Chris,

I use µ/log in conjunction with CloudWatch Insights for many of my projects own projects. I don't personally find the namespaced keywords do be much of an issue. However, there are a number of things you can do to achieve the desired behaviour.

Consider the following:

• µ/log does not mandate the use of namespaced keywords for the event name and for the values. Simple keywords or string would work just fine.
• µ/log publishers, including the CloudWatch one, support custom transformation functions (see https://cljdoc.org/d/com.brunobonacci/mulog/0.9.0/doc/custom-transformations). With the :transform function you can alter the events as you see fit. For example, you could inject a non-namespaced version of the keys and values so that other the discovery function would find them easily.
• CouldWatch Insights supports like matching operator so if you want to use the namespaced keywords, and your event name is :my.namespaced/event1 you easily search them as:

  filter `mulog/event-name` like `event1`

• A useful CloudWatch Insights query to explore which events are available is:

  stats count(*) by `app-name`, `mulog/event-name`

The objective of µ/log is to capture the data and dispatch it to one or more destinations. However, I tried my best to give the users control over which data is published and, at some extent, in which shape.

I hope this help.

[cch1]

It does -I should have realized that I can transform the keys as well as filter the events. That will suffice nicely since foregoing namespaced keys is a non-starter for me.