robinhood / faust

Python Stream Processing
Other
6.7k stars 538 forks source link

faust overrides global logging settings -- unable to use structlog #768

Closed philipmat closed 1 year ago

philipmat commented 1 year ago

Checklist

Steps to reproduce

I'm trying to use structlog inside a Faust app because I want to have JSON output. However, after Faust initialization, it seems all log messages get prefixed by [date time] [1] [WARNING].

import faust
import structlog

structlog.configure(processors=[structlog.processors.JSONRenderer()])
logger = structlog.get_logger()
logger.info("structlog configured", message="hello world")

# set up faust
app = faust.app(...)
topic = app.topic(....)

@app.topic(topic)
async def read_topic():
    logger.info("Reading from topic", topic=topic)
...

Running with `faust -l INFO --app main.app worker

Expected behavior

The first logger.info, on line 6 correctly outputs JSON:
{ "event": "structlog configured", "message": "hello world" } I expected the second logger.info inside read_topic to also output JSON on a line by itself:
{ "event": "Reading from topic", "topic": "test-topic" },

Actual behavior

Instead all logging after Faust starts get prefixed by what looks to be Faust config override, for example:
[2022-11-08 15:16:17,123] [1] [WARNING] { "event": "Reading from topic", "topic": "test-topic" }

Versions

philipmat commented 1 year ago

Closing -- moved to faust-streaming

serpiente commented 4 months ago

Hi @philipmat, did you solve this?