airtai / faststream

FastStream is a powerful and easy-to-use Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis.
https://faststream.airt.ai/latest/
Apache License 2.0
3.18k stars 162 forks source link

Bug: CLI is incompatible with structlog #1900

Closed pySilver closed 2 weeks ago

pySilver commented 2 weeks ago

Describe the bug There is inconsistent behavior when using structlog as described at https://faststream.airt.ai/latest/getting-started/logging/?h=structlog#structlog-example

When app is executed as python my_app.py and logger is configured as logger = structlog.get_logger() we will get consistent formatting and all other benefits of using structlog. However this way we will lose benefits of using faststream cli, such as controllable log-level, realoads, workers etc.

On the other hand, when using faststream cli with logger configured as logger = structlog.get_logger() we will not be able to control log level as in that case checks within faststream.cli.utils.logs.set_log_level will fail: https://github.com/airtai/faststream/blob/fc61e913cea6d09b25524141af7272496606a20f/faststream/cli/utils/logs.py#L67 as structlog's BoundLogger has no such method as setLevel.

Solution I don't have a good solution for that. We may want to modify set_log_level function so it will check if logger is some other type than logging.Logger and call logging.getLogger().setLeveL(level) to configure root logger level. But that might not be a perfect solution I suppose.

pySilver commented 2 weeks ago

alternatively CLI might set env FASTSTREAM_LOG_LEVEL variable so app can handle it manually when needed

Lancetnik commented 2 weeks ago

It doesn't look as FastStream problem for me, please open an Issue to structlog itself to add setLevel method for regular logging compatibility

pySilver commented 2 weeks ago

@Lancetnik not an issue. Here is an quick example of correct integration for others that may bump into this ticket. It is highly recommended to read this official structlog docs section first: https://www.structlog.org/en/stable/standard-library.html#rendering-using-structlog-based-formatters-within-logging

import logging.config

import structlog

from faststream import FastStream
from faststream.nats import NatsBroker

def configure_logging() -> None:
    common_processors = (
        structlog.processors.format_exc_info,
        structlog.stdlib.add_log_level,
        structlog.stdlib.add_logger_name,
        structlog.contextvars.merge_contextvars,
        structlog.stdlib.ExtraAdder(),
        structlog.dev.set_exc_info,
        structlog.processors.TimeStamper(fmt="%Y-%m-%d %H:%M:%S.%f", utc=True),
        structlog.processors.dict_tracebacks,
        structlog.processors.CallsiteParameterAdder(
            (
                structlog.processors.CallsiteParameter.FUNC_NAME,
                structlog.processors.CallsiteParameter.LINENO,
                structlog.processors.CallsiteParameter.FILENAME,
            )
        ),
    )
    structlog_processors = (
        structlog.processors.StackInfoRenderer(),
        structlog.stdlib.PositionalArgumentsFormatter(),
        structlog.processors.UnicodeDecoder(),
        structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
    )
    logging_processors = (structlog.stdlib.ProcessorFormatter.remove_processors_meta,)
    logging_console_processors = (
        *logging_processors,
        structlog.dev.ConsoleRenderer(colors=True),
    )

    handler = logging.StreamHandler()
    handler.set_name("default")
    handler.setLevel(logging.INFO)
    console_formatter = structlog.stdlib.ProcessorFormatter(
        foreign_pre_chain=common_processors,
        processors=logging_console_processors,
    )
    handler.setFormatter(console_formatter)

    handlers = [handler]

    logging.basicConfig(handlers=handlers, level=logging.INFO)
    structlog.configure(
        processors=common_processors + structlog_processors,
        logger_factory=structlog.stdlib.LoggerFactory(),
        wrapper_class=structlog.stdlib.BoundLogger,
        cache_logger_on_first_use=True,
    )

logger = logging.getLogger(__name__)

def app() -> FastStream:
    configure_logging()

    broker = NatsBroker(logger=logger)
    return FastStream(broker, logger=logger)

Credits: @draincoder