search

GoogleCloudPlatform / spring-cloud-gcp

New home for Spring Cloud GCP development starting with version 2.0.
Apache License 2.0
414 stars 308 forks source link

In Google Cloud Logging I'd like to emit structured json via com.google.cloud.spring.logging.LoggingAppender #2428

Open wleese opened 9 months ago

wleese commented 9 months ago

Is your feature request related to a problem? Please describe. When setting up logback.xml to use com.google.cloud.spring.logging.LoggingAppender, a logger.info(myJsonString) will lead to a log entry in the Google Cloud Logging UI, with a message field containing the value of myJsonString.

When this value is JSON and will always be JSON, then putting the value of myJsonString in a message field results in a sub-optimal experience in Google Cloud Logging UI. The value of message will not be recognized as JSON, and will not benefit from being treated as a structured log (no nifty grey triangles to expand/collapse fields within the JSON).

Describe the solution you'd like I'd like the ability to configure com.google.cloud.spring.logging.LoggingAppender so that its input is considered JSON at all times, having it use the log event message as the payload.

Describe alternatives you've considered Alternatively, I can create my own log appender, but this is wasteful for the relatively small change this is likely to be. I also looked into how a string value could be considered a structural log (json) using the Google Cloud Logging UI, but this doesn't seem possible.

mpeddada1 commented 9 months ago

@wleese Thanks for the request! The "Log Via Console" section in the documentation page contains a bit of information on logging JSON entries to the console which may be relevant to your use case?

wleese commented 9 months ago

@mpeddada1 thanks for the suggestion. I was aware of this feature, however due our multi tenant gke cluster with various kinds of jvm and non jvm applications, we run into the issue of duplicating logs because we’re already pushing all console logging for all apps to gcloud logging.

So in short, we could do this, but all jvm apps that use this would have their logs ingested twice.

wleese commented 9 months ago

@mpeddada1 ah, not duplicated now that I think about it. But the ingesting components would need to deal with both structured and unstructured logs. In addition, one would not have a great experience with kubectl show logs due to the json.

Update: one more reason that finally comes to mind: as mentioned we have a multi tenant gke cluster in 1 gcloud project. We do some usage/cost accounting to “bill” our users. By relying on console logs -> 1 single gcloud project’s logging api / logsink->pubsub->ELK stack, it becomes a bit harder to do usage/cost accounting.