Override name of log record field

[sshishov]

I would like to request feature to being able to override log record attribute using setting.

We use kibana and would like to see something more meaningful rather than cid.

[dbaty]

Hi. Do you mean that you would like your logs to look like:

[correlation_id=f94dccf6-7efa-4970-9da1-ff4ac0dda280] INFO 2019-08-27T15:53:09.498928+00:00 mypackage.mymodule This is a log'

If so, you can do that by tweaking the formatters in your settings.LOGGING. As per the docs. LOGGING looks like this:

LOGGING = {
    'formatters': {
        'verbose': {
            'format': '[cid: %(cid)s] %(levelname)s %(asctime)s %(module)s %(message)s'
        },
        'simple': {
            'format': '[cid: %(cid)s] %(levelname)s %(message)s'
        },
    },
    # and define 'handlers', 'filters' and 'loggers' here...
}

If you'd prefer having "correlation_id" instead of "cid", you can do so by changing the formaŧ key(s), like this:

'[correlation_id=%(cid)s] %(levelname)s %(asctime)s %(module)s %(message)s'

Does that answer your request?

[sshishov]

Hi, I know all this but unfortunately, we do not process log files using FileBeat or any other approach. We are using logstash through TCP and send record directly (I assume it sent as JSON). Therefore we do not have control over the field's name.

I can create PR to illustrate to you what we want. Why we want it? Because we are moving from another similar library: django-log-request-id which lack this functionality as well and it is more difficult to mock it for testing, plus it does not have the ability to chain correlation ids

[dbaty]

I am missing a piece of information in your setup. How does changing the name of the attribute in the LogRecord, affects what is sent by Logstash? Do you use a custom handler that automatically sends all attributes of the LogRecord, something like this:

class Handler:
    def emit(self, record):
        logstash.send(record.__dict__)

That's indeed what python-logstash seems to do: https://github.com/vklochan/python-logstash/blob/a8be251ad9fc102af668c38a5397515b0804da75/logstash/formatter.py#L40-L45.

If so, then your pull request #46 seems right. If you could add some documentation for the setting and a test, that would be great. :)

[sshishov]

@dbaty exactly we are using python-logstash and therefore we do not have ability to override it rather than add our own log filter. It would be nice to have ability to override it. I will try to enhance documentation and add testing as soon as I have time.

Thanks.