Support for other structured log-formats such as LOGFMT

[RovoMe]

As requested I create a new, separate issue. Besides JSON there are other log formats like LOGFMT wich is basically just a format that prints out properties in a key1=value1 key2=value2 key3="value 3" ... manner which therefoe should be rather trivial to parse. There are already quite some libraries for different languages out there which help destructuring all the segments within log lines but I'm not sure if there is actually an RFC or fixed rules out there in regards to nested object data, as JSON i.e. supports it.

I spent some time yesterday to dig through your code and started to work on an own LOGFMT implementation but I still see the benefit of having one plugin that is able to support lots of various structured log formats as then people don't have to install X plugins that basically all do the same thing, which also might be a bit more wasteful in terms of CPU cycles and what not. Besides that, I'm fluent in Java but haven't really worked with Kotlin before and it seems to allow something like global static function without class definitions. I therefore avoided porting the code to Java and instead used Google, StackOverflow ad ChatGPT to get a somehow working first LOGFTM formatter plugin up and running (not committed and deployed or released though yet).

From what I've learned your plugin makes heavy use of JsonNode which therefore might need to get abstracted to a more general type so the formatter can take care of formatting JSON, LOGFMT and what not to come in future. In my testing I did use a simple Map<String, String> structure instead which though means that log lines can't have nested information.

Sample log lines for LOGFMT may look like these:

...
time="2024-09-05T15:51:28.425" level=debug msg="Running with Spring Boot v3.2.6, Spring v6.1.8" package=org.springframework.boot thread=main
time="2024-09-05T15:51:28.425" level=info msg="The following 1 profile is active: \\\"docker\\\"" package=org.springframework.boot thread=main
...
time="2024-09-05T15:51:28.426" level=warn thread=main package=org.acme module=TestApp msg="Some problem encountered" error="java.lang.Exception: null\n\tat org.acme.TestApp.logError(TestApp.java:94)\n\tat org.acme.TestApp.standardLogger(TestApp.java:34)\n\tat org.acme.TestApp.<init>(TestApp.java:20)\n\tat org.acme.TestApp.main(TestApp.java:16)\n"
...
time="2024-09-05T15:51:33.612" level=info msg="\\n\\nError starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled." package=org.springframework.boot.autoconfigure.logging thread=main
time="2024-09-05T15:51:33.621" level=error msg="\\n\\n***************************\\nAPPLICATION FAILED TO START\\n***************************\\n\\nDescription:\\n\\nBinding to target org.acme.Main failed:\\n\\n    Property: some.property\\n    Value: \\\"null\\\"\\n    Reason: must not be null\\n\\n\\nAction:\\n\\nUpdate your application's configuration\\n" package=org.springframework.boot.diagnostics thread=main

For a folded message simply showing time, log-level and the message of the log line, as you do for JSON based log lines is enough I think and when you expand a log line rendering it like

[
    time =  2024-09-05T15:51:28.425
    level = debug
    msg = Running with Spring Boot v3.2.6, Spring v6.1.8
    package = org.springframework.boot
    thread = main
]

would be helpful to distinguish LOGFMT log lines slightly from JSON log lines. Whether nested objects should be supported, I don't know. As there is no definitiv spec here available on what a LOGFMT log line may contain, implementing something out of thin air always has the potential that at one point a real standard evolves and then the own approach needs to be thrown away and reimplemented. In the logback formatter we use to log logfmt to Grafana/Loki we at least do not support nested objects yet and as such our requirements are rather simple :)

[orangain]

@RovoMe Thank you for creating the issue and detailed explanation!

From what I've learned your plugin makes heavy use of JsonNode which therefore might need to get abstracted to a more general type so the formatter can take care of formatting JSON, LOGFMT and what not to come in future. In my testing I did use a simple Map<String, String> structure instead which though means that log lines can't have nested information.

I think we can use the JsonNode as a general type. This is because Jackson supports various formats other than JSON, and the JsonNode class is widely used in these text formats despite its name. Therefore, I think it is reasonable to create something like jackson-dataformat-logfmt and use it.

For a folded message simply showing time, log-level and the message of the log line, as you do for JSON based log lines is enough I think and when you expand a log line rendering it like ... would be helpful to distinguish LOGFMT log lines slightly from JSON log lines.

As you say, the placeholder [...] seems nice, but my concern is that if we support more formats other than json and logfmt, we might run out of types of brackets. Another concern is that the folding behavior of this plugin is achieved by a bit hacky way, so increasing the folding patterns might cause unwanted side effects.

Maybe we should consider another way to solve this for example, show format name somewhere or provide feature to show original (single line) value. It is never easy, though.

---

Finally, I would like to ask you a question. Is there a strong motivation to use logfmt rather than JSON in your development? From my understanding, logfmt's strong point is human-friendly, but if you use this plugin, readability of the log does not differ between logfmt and JSON. As you say, logfmt does not have strict spec, and JSON would be better in terms of interoperability.

This is not a criticism or anything like that, I just want to clarify the motivation to support logfmt in this plugin :pray: