How to include all available fields in a log record + some custom fields?

[nyue]

Currently, I am writing a custom formatter to include all fields with the intention of adding some of my own later.

I have two sets of questions

Q1 : Is this the correct way? i.e. subclass the formatter and than copying field by field over ? or am I going about it the wrong way ?

Ultimately, I will want to include most of the default fields plus I am going to add some custom in-house fields.

class CustomJsonFormatter(jsonlogger.JsonFormatter):

    def add_fields(self, log_record, record, message_dict):
        import datetime
        super(CustomJsonFormatter, self).add_fields(log_record, record, message_dict)

        log_record['args'] = record.args
        # log_record['asctime'] = record.asctime
        log_record['created'] = record.created
        log_record['exc_info'] = record.exc_info
        log_record['exc_text'] = record.exc_text
        log_record['filename'] = record.filename
        log_record['funcName'] = record.funcName
        log_record['levelname'] = record.levelname
        log_record['levelno'] = record.levelno
        log_record['lineno'] = record.lineno
        log_record['module'] = record.module
        log_record['msecs'] = record.msecs
        log_record['message'] = record.message
        log_record['msg'] = record.msg
        log_record['name'] = record.name
        log_record['pathname'] = record.pathname
        log_record['process'] = record.process
        log_record['processName'] = record.processName
        log_record['relativeCreated'] = record.relativeCreated
        log_record['stack_info'] = record.stack_info
        log_record['thread'] = record.thread
        log_record['threadName'] = record.threadName

Q2 : What if I want most but not all the default fields, is there some pythonic way to do that ?

[nhairs]

I've not yet tested this code, but here's a starting point for doing both:

Note: this code is written for logging.LogRecord which assume that all data is in __dict__ already.

class CustomJsonFormatter(jsonlogger.JsonFormatter):

    def add_fields(self, log_record, record, message_dict):
        super(CustomJsonFormatter, self).add_fields(log_record, record, message_dict)

        ## Add Everyting
        for name, value in record.__dict__.items():
            if name not in log_record and not name.startswith("_"):
                # Only log names that have not already been added and aren't some
                # "private" attribute.
                log_record[name] = value

        ## Exclude
        # TODO: populate self.excluded_fields
        # self.excluded_fields = ["foo", "bar", "threadName"]

        for name in self.excluded_fields:
            if name in log_record:
                del log_record[name]

        return

Edit

It might be possible to do this by setting reserved_attrs=tuple() agument, excluding specific fields can then be done by adding them back in to the reserved_attrs.

Note: this won't work for "calculated" attributes such as message, asctime, exc_info without first adding the attributes to _required_fields (via parsinging the format).