fluent / fluent-logger-python

A structured logger for Fluentd (Python)
http://fluentd.org/
Other
444 stars 138 forks source link

Formatting message does not work #190

Open hemanthactionfi opened 2 years ago

hemanthactionfi commented 2 years ago

While trying to truncate the message part using formatter. fluent.handler.FluentRecordFormatter works normally with all fields except message

configuration

LOGGING = {
    'version': 1,
    'disable_existing_loggers': True,
    'formatters': {
        'verbose': {
            'format': "[%(levelname)s %(asctime)s %(module)s -> %(lineno)d] %(message)s",
            'datefmt': "%Y-%b-%d %H:%M:%S"
        },
        'console': {
            'format': '[%(levelname)s %(asctime)s] %(message)s',
            'datefmt': "%Y-%b-%d %H:%M:%S"
        },
        'fluent_fmt': {
            '()': "fluent.handler.FluentRecordFormatter",
            'format': {
                'lvl': '%(levelname)s',
                'host': '%(hostname)s',
                'mod': '%(module)s',
                'line': '%(lineno)d',
                'tms': '%(created)d',
                'trace': '%(exc_text)s',
                "proc": '%(processName)s',
                "message": "%(message).5s"
            }
        },
    },
    'handlers': {
        'console': {
            'level': 'INFO',
            'class': 'logging.StreamHandler',
            'formatter': 'verbose'
        },
        'file': {
            'level': 'INFO',
            'class': 'logging.handlers.TimedRotatingFileHandler',
            'filename': 'celery_nohup',
            'when': 'D',  # this specifies the interval
            'interval': 1,  # defaults to 1, only necessary for other values
            'formatter': 'verbose',
        },
        'fluentd': {
            'level': 'INFO',
            'class': 'fluent.handler.FluentHandler',
            'formatter': 'fluent_fmt',
            'tag': 'default.log',
            'host': 'localhost',
            'port': 24224,
            'timeout':3.0,
            'verbose': False,
            "msgpack_kwargs":{'default' : str}
        }, 

    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'level': "INFO",
            'propagate': True,
        },
        'celery': {
            'handlers': ['console', 'fluentd'],
            'level': "INFO",
            'propagate': True,
        },
        'ap_scheduler': {
            'handlers': ['console', 'fluentd'],
            'level': "INFO",
            'propagate': True,
        },
    },
}

expected output: message truncated to 5 characters fluentd_1 | 2022-06-02 18:58:43.000000000 +0000 default.log: {"lvl":"INFO","host":"E7440","mod":"mingle","line":"40","tms":"1654196323","trace":"None","proc":"MainProcess","message":"mingl"}

obtained output fluentd_1 | 2022-06-02 18:58:43.000000000 +0000 default.log: {"lvl":"INFO","host":"E7440","mod":"mingle","line":"40","tms":"1654196323","trace":"None","proc":"MainProcess","message":"mingle: searching for neighbors"}