Open Timelessprod opened 1 month ago
Hi @Timelessprod,
Just so you're aware it looks like python-json-logger is currently unmaintained, that said I am working on a maintained fork.
Whilst the below examples have been tested using my fork, I'm pretty sure that this still applies to the original library as well.
Any field that you wish to rename that a standard LogRecord attribute needs to be included in the format of the Formatter.
import logging
# https://github.com/nhairs/python-json-logger v3.1.0.rc2
from pythonjsonlogger.json import JsonFormatter
## Setup
## -------------------------------------
logger = logging.getLogger("test")
logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
logger.addHandler(handler)
## Problem
## -----------------------------------------------------------------------------
formatter = JsonFormatter(
rename_fields={"asctime": "logstash_asctime", "levelname": "logstash_levelname"}
)
handler.setFormatter(formatter)
logger.info("this does not bring me joy")
# --- Logging error ---
# Traceback (most recent call last):
# File "/usr/lib/python3.10/logging/__init__.py", line 1100, in emit
# msg = self.format(record)
# File "/usr/lib/python3.10/logging/__init__.py", line 943, in format
# return fmt.format(record)
# File "/home/nhairs/git/self/python-json-logger/src/pythonjsonlogger/core.py", line 234, in format
# self.add_fields(log_record, record, message_dict)
# File "/home/nhairs/git/self/python-json-logger/src/pythonjsonlogger/core.py", line 312, in add_fields
# self._perform_rename_log_fields(log_record)
# File "/home/nhairs/git/self/python-json-logger/src/pythonjsonlogger/core.py", line 317, in _perform_rename_log_fields
# log_record[new_field_name] = log_record[old_field_name]
# KeyError: 'asctime'
# Call stack:
# File "/home/nhairs/git/scrap/python_json_logger_190.py", line 18, in <module>
# logger.info("this does not bring me joy")
# Message: 'this does not bring me joy'
# Arguments: ()
## Solution
## -----------------------------------------------------------------------------
formatter = JsonFormatter(
"{message}{asctime}{levelname}",
style="{",
rename_fields={"asctime": "logstash_asctime", "levelname": "logstash_levelname"}
)
handler.setFormatter(formatter)
logger.info("this brings me joy")
# {"message": "this brings me joy", "logstash_asctime": "2024-05-14 20:13:39,101", "logstash_levelname": "INFO"}
Note: that the asctime
field format is controlled by the datefmt
argument not the JSON encoder.
There's a few ways that fields can be added depending on your exact use case.
When making a logging call you can pass in the extra
argument.
logging.info("some message", extra={"some_id": get_my_id()})
You can override the process_log_record
method:
class MyFormatter(JsonFormatter):
def process_log_record(log_record):
log_record["some_id"] = get_my_id()
return log_record
You can set static_fields
:
formatter = JsonFormatter(static_fields={"service_name": "my_service"})
You can attach data to the LogRecord
object using a filter, the additional LogRecord
attributes will automatically be logged. https://docs.python.org/3/howto/logging-cookbook.html#using-filters-to-impart-contextual-information
Hello,
I'm using your library and would need to set it up to output logs in the logstash format so they get parsed correctly by third party services using logstash. for example the current
asctime
andlevelname
don't get recognized.I was tempted to use https://github.com/mbarrientos/logstash-python-formatter but it's not maintained for too long now and this may cause security problems.
I read the README and its section about adding extra fields but it seems to be outdated as I can'T see this fields in the class builder of
JsonFormatter
. Could you please guide me on what to do to change to rename fields and add new ones ?Thanks!