Support for custom format/fields in quarkus-logging-json

[overthetop]

I'm using the quarkus-logging-json artifact. I'm looking for a way to customize the json logging format. Our case is that we are using ELK stack for logging and all of our services (spring boot) are logging in the stdout entries formatted with the logstash-logback-encoder

https://github.com/logstash/logstash-logback-encoder

I want to output logs with the same format with Quarkus

For example in my case these fields:

@timestamp | Time of the log event. (yyyy-MM-dd'T'HH:mm:ss.SSSZZ) See customizing timestamp. @version | Logstash format version (e.g. 1) See customizing version. message | Formatted log message of the event logger_name | Name of the logger that logged the event thread_name | Name of the thread that logged the event level | String name of the level of the event level_value | Integer value of the level of the event stack_trace | (Only if a throwable was logged) The stacktrace of the throwable. Stackframes are separated by line endings.

if you refer to the https://github.com/logstash/logstash-logback-encoder readme, you'll see how much flexible it is. I believe that many organizations will need to be able to control the logs format and add custom fields as well.

[argenstijn]

+1

[alainmucyo]

Any update on the issue?

[loicmathieu]

@alainmucyo you can try https://github.com/quarkiverse/quarkus-logging-json that support structured logging. I think we can close this issue as this will not be supported with the core Quarkus logmanager.

[argenstijn]

Personal view..this should be supported by the core log module. Why not?

Outlook voor Androidhttps://aka.ms/AAb9ysg downloaden

---

From: Loïc Mathieu @.> Sent: Monday, July 26, 2021 9:57:56 AM To: quarkusio/quarkus @.> Cc: argenstijn @.>; Comment @.> Subject: Re: [quarkusio/quarkus] Support for custom format/fields in quarkus-logging-json (#7451)

@alainmucyohttps://github.com/alainmucyo you can try https://github.com/quarkiverse/quarkus-logging-json that support structured logging. I think we can close this issue as this will not be supported with the core Quarkus logmanager.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/quarkusio/quarkus/issues/7451#issuecomment-886467508, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAA75LDLBNSXQS3CNPWQ2ZLTZUIQJANCNFSM4K4XTGHA.

[ayhanap]

With the #23977 now merged, now I can change the default level field in json logging to severity as it is required by Google Cloud Platform structured logging (https://cloud.google.com/logging/docs/structured-logging)

1. added maven dependency for quarkus-logging-json
2. added quarkus.log.console.json.key-overrides=level=severity in application.properties to map change level to severity
3. Bonus: You can also add %dev.quarkus.log.console.json=false and %test.quarkus.log.console.json=false to disable json logging in test and dev env.

After these steps, now my logs in GCP shows as debug, info, warn, error etc. instead of showing as default.

[loicmathieu]

@ayhanap this would be great if you can provide a PR to document these steps in our Deploying to Google Cloud guide. This would help others ;)

[ayhanap]

Can do that, but sadly with the current options I can only fix severity field. GCP also also requires stacktrace to be inside message field (after the message) to mark it for ErrorReporting. Which can only be done with custom jsonproviders with https://github.com/quarkiverse/quarkus-logging-json. quarkiverse/quarkus-logging-json has support for aws json template but there is an open PR to support GCP(with other optional fields) which needs some clarifaction and work.

@loicmathieu Do you think we should document how it can be done with the quarkus-logging-json extension? (Maybe after the PR on that merged, or the way @BeneStem mentioned)

[loicmathieu]

@ayhanap better wait for the openned PR to be merged.

[Sangames7]

2. override

@ayhanap Is there a way to override the key of a nested json? For example, consider below json:

{"timestamp":"2022-10-25T15:46:58.581+05:30","sequence":2067,"loggerClassName":"org.jboss.logging.Logger","loggerName":"org.acme.opentelemetry.TracedResource","level":"INFO","message":"INFOOOO","threadName":"executor-thread-0","threadId":138,"mdc":{"spanId":"c59f47e85470c1db","traceId":"cffd575d440cc9eb2bda5265b38978c7","sampled":"true","transaction_id":"1"},"ndc":"","hostName":"wsamzn-o6p0oke8","processName":"opentelemetry-quickstart-dev.jar","processId":11456,"service.name":"myservice"}

I need to change the spanId inside mdc to span.id.

[paomian]

Any update ？ I want add field in log by log API? now the log can just add constant value by config. how to add custome field by log API?

[ericsouza]

Hi, how can I format my json logging to use GELF format? I'm collecting logging direct from my pods, because of this using the quarkus-logging-gelf extensions is an overhead for my case. Also, the logstash-gelf library (used by quarkus-logging-gelf) is now archived

[mhagnumdw]

Is it currently possible to print the entire stacktrace just separated by line breaks in the exception field instead of fragmented into frames?