Nebo15 / logger_json

JSON logger formatter with support for Google Cloud, DataDog and other for Elixir.
https://nebo15.github.io/logger_json/
MIT License
237 stars 92 forks source link

Suggestions on how to set google logger logName? #62

Closed patmaddox closed 3 years ago

patmaddox commented 3 years ago

Do you have any suggestions on how we could go about setting the logName attribute per LogEntry documentation? I don't believe this library currently supports that - and before I fork and commit to an approach, I wanted to see how you'd consider going about this.

One possibility is just to look for the log_name metadata keyword and move it to the appropriate place in the google log structure.

Logger.info("my log message", log_name: "projects/myproject/logs/custom-log-name") would produce:

"jsonPayload":{
    "message":"my log message",
    ...
  },
"logName":"projects/myproject/logs/custom-log-name",

(as opposed to the current behavior which puts logName inside of the jsonPayload)

But it's not the only google-logging specific keyword that people might want to set, so perhaps we want a special metadata struct to hold on to google-specific keywords. Something like Logger.info("my log message", _google: [log_name: "projects/myproject/logs/custom-log-name"]) to produce the same JSON structure as above.

Another possibility is to not do any of this in LoggerJSON and instead wrap GoogleCloudLogger and modify the data structure in the wrapper module.

AndrewDryga commented 3 years ago

Hey @patmaddox, can you please give a little detail on how do you use this field? Do you set it to be static per application or you need to pass it when you call Logger.* macros?

patmaddox commented 3 years ago

We are still working that out. Our first approach will most likely be to pass it as metadata to the Logger.* macros. We may eventually refactor this specific logging functionality to its own process or application.

Our use case is we're creating an audit log, logging to google cloud logging. Currently we add audit_log: true metadata, and have a log sink that saves those specific logs to a custom log bucket. We would prefer to set a custom logName so it appears specifically in the log name list:

image

So most of our app will log to stdout as normal, and we'd like our audit log to also log to stdout but also set the logName JSON property.

AndrewDryga commented 3 years ago

@patmaddox I checked the docs and It doesn't look like there is a way to set that field, the primary documentation is here: https://cloud.google.com/logging/docs/agent/configuration#special-fields. LogEntry has many fields but they are not available as everything in GKE still goes through stackdriver-fluentd which only supports those mentioned in a doc. So I guess the only way to set it would be to alter fluentd and manage it on your own, but It might it won't worth the effort for you

AndrewDryga commented 3 years ago

I'm closing this for now as this is more like a GCL limitation than something we can do from a library.