hynek / structlog

Simple, powerful, and fast logging for Python.
https://www.structlog.org/
Other
3.48k stars 220 forks source link

How to use structlog with logfmt formatted logs? #625

Closed 5er9e1 closed 3 months ago

5er9e1 commented 4 months ago

I want to print out root logs in logfmt format. structlog should print logs in the same format. I tried this example and looks like it doesn't work with logfmt.

Here is a first try:

import logging
import sys

import logfmter
import structlog

handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(logfmter.Logfmter())
root_logger = logging.getLogger()
root_logger.addHandler(handler)

structlog.get_logger("test").warning("hello")
logging.getLogger("test").warning("hello")

It prints:

2024-06-04 22:11:01 [warning  ] hello                         
at=WARNING msg=hello

Here is structlog is definitely not a logfmt formatted.

After this I tried to add render:

import logging
import sys

import logfmter
import structlog

handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(logfmter.Logfmter())
root_logger = logging.getLogger()
root_logger.addHandler(handler)

structlog.configure(
    processors=[
        structlog.stdlib.filter_by_level,
        structlog.stdlib.add_logger_name,
        structlog.stdlib.add_log_level,
        structlog.stdlib.PositionalArgumentsFormatter(),
        structlog.processors.TimeStamper(fmt='iso'),
        structlog.processors.StackInfoRenderer(),
        structlog.processors.format_exc_info,
        # ADD LOGFMT RENDER
        structlog.processors.LogfmtRenderer(),
    ],
    logger_factory=structlog.stdlib.LoggerFactory(),
    wrapper_class=structlog.stdlib.BoundLogger,
    cache_logger_on_first_use=True,
)

structlog.get_logger("test").warning("hello")
logging.getLogger("test").warning("hello")

And fail again:

at=WARNING msg="event=hello logger=test level=warning timestamp=2024-06-04T13:12:54.637931Z"
at=WARNING msg=hello

Obviously correct log line should be like:

at=WARNING msg=hello logger=test level=warning timestamp=2024-06-04T13:12:54.637931Z

Is it possible to format both structlog and logging as logfmt?

hynek commented 3 months ago

It looks like you expect stdlib output to match structlog's output? Unfortunately that means that you have to teach those two to cooperate (or, as pointed out in https://www.structlog.org/en/stable/standard-library.html#suggested-configurations, configure them in their own ways to have the same output).

hynek commented 3 months ago

I assume https://stackoverflow.com/questions/78575692/how-to-use-structlog-with-logfmt-formatted-logs-in-python/78681766#78681766 is you too so closing.