loki4j / loki-logback-appender

Fast and lightweight implementation of Logback appender for Grafana Loki
https://loki4j.github.io/loki-logback-appender/
BSD 2-Clause "Simplified" License
314 stars 27 forks source link

[Enhancement request] Option to create MDC format #201

Closed patpatpat123 closed 10 months ago

patpatpat123 commented 11 months ago

Hello team,

I would like to reach out with a small enhancement request.

Nowadays, MDC format logging is becoming increasingly popular. With that, I was wondering if this project could provide a simple way to turn on and off MDC format logs.

I think one can manipulate the pattern manually and create something like:

<message>
            <pattern>
{ 
    @timestamp:  ${timestamp}
    @version:  ${version}
...
}
</pattern>
        </message>

But would it be possible to have that directly?

Thank you

nehaev commented 11 months ago

Hey @patpatpat123, not sure I get your idea completely. Loki4j has MDC support, please check out the docs. The example there is for labels, but for messages it works exactly the same way, because both of them are standard Logback patterns.

If that's not what you meant, please let me know.

patpatpat123 commented 11 months ago

Hello @nehaev ,

As of right now, it seems that in order to see MDC logs, one needs to configure something like:

    <appender name="LOKI" class="com.github.loki4j.logback.Loki4jAppender">
        <http>
            <url>${HOST}</url>
        </http>
        <format>
            <label>
                <pattern>name=${name},host=${HOSTNAME},level=%level,pid=${PID},thread=%thread</pattern>
                <readMarkers>true</readMarkers>
            </label>
            <message>
                <pattern>
{
"name": "${name}"
"pid": "${PID}",
"level": "%level",
"thread": "%thread",
"class": "%logger{36}",
"traceId": "%X{traceId}"
"spanId": "%X{spanId}"
"message": "%message"
}
                </pattern>
            </message>
            <sortByTime>true</sortByTime>
        </format>
        <verbose>false</verbose>
    </appender>

As shown in this tutorial: https://piotrminkowski.com/2023/07/05/logging-in-spring-boot-with-loki/

This requires the user to maintain the pattern block manually:

{
   "level":"%level",
   "class":"%logger{36}",
   "thread":"%thread",
   "message": "%message",
   "requestId": "%X{X-Request-ID}"
}

While other loggers can offer an easy configurable option;

<configuration>
    <appender name="jsonConsoleAppender" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
    </appender>
    <root level="INFO">
        <appender-ref ref="jsonConsoleAppender"/>
    </root>
</configuration>
logging.config=classpath:logback-structured.xml

I believe this would benefit loki users, to have an easier default MDC format integration, without having to maintain the MDC key values.

Thank you

nehaev commented 10 months ago

Thanks for the clarification! Makes sense now.

Probably this could only be possible with JSON message layout.

nehaev commented 10 months ago

Closing in favor of #205