madzak / python-json-logger

Json Formatter for the standard python logger
BSD 2-Clause "Simplified" License
1.7k stars 231 forks source link

[HELP] Logging Custom Python Class #108

Closed TWDickson closed 1 year ago

TWDickson commented 3 years ago

I have a custom class that I'm trying to log:

class NameMatchDiagnostic():

    def __init__(self):
        self.row_data = {"key", "value"}
        self.table = "string"
        self.action = "string"
        self.db_record = {"key", "value"}
        self.top_matches = [{"key", "value"}, {"key", "value"}, {"key", "value"}]

All in all very basic class, I'm pulling out my hair trying to get this thing to work, ideally I'm trying to log a timestamp, message, and then this data.

Any help would be greatly appreciated.

radder5 commented 3 years ago

I know this is a while after you first posted, but I had a similar requirement. My solution was to create a pydantic BaseModel subclass just to hold the data (as you example) - but I guess you don't need to use pydantic (it's just a data structure). Then set up logging in usual way using an out of the box formatter like

json:
    (): pythonjsonlogger.jsonlogger.JsonFormatter
    format: '%(asctime)s - %(name)s - %(threadName)s - %(module)s - %(funcName)s - %(lineno)s - %(levelname)s - %(message)s'

which gives you timestamp etc. The message is essentially what you log e.g.

class MyData(BaseModel):
    id: str
    prop1: str
    prop2: Dict[str, str]
    prop3: List[Dict[str, str]]

the bit that took me ages to sort was how to just do

logger.info(MyData())
or
logger.info(MyData(id='1234',prop1='hello',prop2={'one':'1'})

but that is done using a Filter in my case

class BaseModelToDictFilter(Filter):
    """
    This is a filter which converts a msg BaseModel (or subclass) into a dictionary
    """
    def filter(self, record):
        if isinstance(record.msg, MyData):
            record.msg = record.msg.dict(exclude_none=True)
        return True

I actually create a base log class and extend for all other logging 'things' then configure filter in filters property HTH