search

madzak / python-json-logger

Json Formatter for the standard python logger
BSD 2-Clause "Simplified" License
1.7k stars 231 forks source link

Jsonformatter formats the newline character #115

Closed anandtripathi5 closed 2 years ago

anandtripathi5 commented 3 years ago

Jsonformatter will not format the newline character that is inside the message fields too.

Ex: without this commit { "rid": "6c0320a4-ed87-4d0a-a668-b7ceef94eecf", "asctime": "2021-03-23 14:13:50,151", "filename": "", "module": "", "funcName": "", "lineno": 16, "levelname": "INFO", "message": "log line 1\nlog line 2" }

After this commit { "rid": "6c0320a4-ed87-4d0a-a668-b7ceef94eecf", "asctime": "2021-03-23 14:13:50,151", "filename": "", "module": "", "funcName": "", "lineno": 16, "levelname": "INFO", "message": "log line 1 log line 2" }

anandtripathi5 commented 3 years ago

I am facing this problem as we are using this library in our organization. Please let me know when we can merge this or when a new version will be available of this. @madzak

Thanks

anandtripathi5 commented 3 years ago

@madzak @svisser can please check this pull request!!

anandtripathi5 commented 3 years ago

This will not produce an invalid JSON as I have implemented this by inheriting this library and using that in ELK stack and in cloud watch. This will change only if the text inside the JSON is having new line character nothing else

On Fri, 28 May, 2021, 12:51 am Simeon Visser, @.***> wrote:

@.**** commented on this pull request.

In src/pythonjsonlogger/jsonlogger.py https://github.com/madzak/python-json-logger/pull/115#discussion_r640903852 :

@@ -179,7 +179,7 @@ def jsonify_log_record(self, log_record):

 def serialize_log_record(self, log_record):
     """Returns the final representation of the log record."""
  • return "%s%s" % (self.prefix, self.jsonify_log_record(log_record))
  • return "%s%s" % (self.prefix, self.jsonify_log_record(log_record).replace('\n', '\n'))

Producing valid JSON is key here, and in addition to the above point, this PR changes the behaviour for all users regardless of whether they actually want that. But producing invalid JSON is not something that we should do anyway.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/madzak/python-json-logger/pull/115#discussion_r640903852, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACR2BGW26BUXOEIAOH36AO3TP2LUNANCNFSM4ZUXEJMA .

svisser commented 3 years ago

It's a good thing that the change works for you but, in general, this library needs to output JSON-formatted log lines, so the return value of format() should be a single line, as JSON. If we run .replace('\\n', '\n') on everything, we'd format it as multiple lines.

anandtripathi5 commented 3 years ago

Can we introduce a config value in library. If user entered it as NEW_LINE_FORMATTER=true then only enable this feature. If that works for you!

On Fri, 28 May, 2021, 3:23 pm Simeon Visser, @.***> wrote:

It's a good thing that the change works for you but, in general, this library needs to output JSON-formatted log lines, so the return value of format() should be a single line, as JSON. If we run .replace('\n', '\n') on everything, we'd format it as multiple lines.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/madzak/python-json-logger/pull/115#issuecomment-850299922, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACR2BGRMWCSC4KYCSVDPJALTP5RZZANCNFSM4ZUXEJMA .

svisser commented 3 years ago

What if you import JsonFormatter from this library, subclass it and make the change in the subclass? I'd be more in favor of that solution than letting the library output lines which aren't single lines.

madzak commented 2 years ago

As discussed above, we don't want to break the JSON format and still honor the "logging" format which is outputting the full JSON object per message (line).