cmanaha / python-elasticsearch-logger

Python Elasticsearch handler for the standard python logging framework
Other
232 stars 116 forks source link

Message is empty #9

Closed hstadler closed 7 years ago

hstadler commented 7 years ago

Hi,

I'm having a problem using args in log calls, as this leads to an empty message (the log record seems not have this attribute) when this is the only logging handler. For example a log.info('The value is: %s', some_value) gives me an empty message and msg just holds the log string The value is: %s (without the args substituted).

Not sure if I'm using it wrong or how to get the whole log message transmitted. I would be glad if you can give me a hint. Thx!

I'm using logging.config.dictConfig to configure logging with the following dict:

{
    "version": 1,
    "incremental": false,
    "formatters":
    {
        "simple":
        {
            "format": "%(message)s"
        }
    },
    "handlers":
    {
        "elasticsearch":
        {
            "class": "cmreshandler.cmreshandler.CMRESHandler",
            "level": "DEBUG",
            "formatter": "simple",
            "hosts": [
                {
                    "host": "127.0.0.1",
                    "port": 9200
                }
            ],
            "auth_type": "ext://cmreshandler.cmreshandler.CMRESHandler.AuthType.NO_AUTH",
            "es_index_name": "myapp"
        }
    },
    "root":
    {
        "level": "DEBUG",
        "handlers": ["elasticsearch"]
    }
}
cmanaha commented 7 years ago

just a quick heads up. I'm trying to reproduce this issue and find the cause. Updated: Just validated my assumption. The issue is not with the configuration; The issue is in the logging line.

log.info('The value is: %s', some_value)

is not correct. The following are correct:

log.info('The value is: %s' % some_value) or log.info("The value is: {0}".format(some_value))

Additionally, for this handler you do not need a formatter; Those sections can be deleted from the configuration

hstadler commented 7 years ago

Thx for your reply and sorry for this late response:

I don't think the syntax log.info('The value is: %s', some_value) is incorrect, because it is described in the Python 2.7 logging documentation and howtos. This is relevant (I think not only) for me, because I use libraries like SQLAlchemy, etc. that use that syntax too.

For me a workaround using a filter that creates the message mimics the behaviour:

class CMRESFixFilter(logging.Filter):
    def filter(self, record):
        if not hasattr(record, 'message'):
            record.message = record.getMessage()
        return 1

and adding 'args' to __LOGGING_FILTER_FIELDS as the serialization might fail (TypeError: Unable to serialize xyz) when the log record is sent.