bobbui / json-logging-python

Cloud-native distributed Python logging library to emit JSON log that can be easily indexed by logging infrastructure
Apache License 2.0
303 stars 62 forks source link

logger is logging multiple times #71

Closed bluebrown closed 3 years ago

bluebrown commented 3 years ago

Hi, I am using flask and the logger is logging up to 4 times depedning on what im doing there. All i want is to set some configuration. It seems like every interaction with the logging module creates another handler or something.

app = flask.Flask(__name__)
json_logging.init_flask(enable_json=True)
json_logging.init_request_instrument(
    app, exclude_url_patterns=[r'/ready', r'/alive'])

app.logger.setLevel(logging.getLevelName(env.get('LOGLEVEL', 'INFO').upper()))
app.logger.addHandler(logging.StreamHandler(sys.stdout))
app.logger.removeHandler(default_handler)

app.config["MONGO_URI"] = env.get("MONGO_DSN")

client = MongoClient(app.config["MONGO_URI"])
app.logger.debug(client.server_info())
bobbui commented 3 years ago

@bluebrown can u provide a fully working code sample

bluebrown commented 3 years ago

That is a working example, all what's missing is the import statements., which I though I can leave out for brevity as they are apparent.

I made it even smaller now.

import flask
from os import environ as env
from pymongo import MongoClient
import json_logging
import logging

app = flask.Flask(__name__)
json_logging.init_flask(enable_json=True)
json_logging.init_request_instrument(
    app, exclude_url_patterns=[r'/ready', r'/alive'])

app.logger.setLevel(logging.getLevelName(env.get('LOGLEVEL', 'INFO').upper()))

app.config["MONGO_URI"] = env.get("MONGO_DSN")

client = MongoClient(app.config["MONGO_URI"])
app.logger.debug(client.server_info())

I m doing further testing and I noticed something. When I run this via the below I dont seem to get duplicates.

python -m flask run --host 0.0.0.0

When I use gunicorn it will duplicate but only in the entry point. Logs from other modules than the entrypoint are not double.

gunicorn \
    --worker-tmp-dir=/dev/shm \
    --workers=2 \
    --threads=4 \
    --worker-class=gthread  \
    --bind=0.0.0.0:5000 \
    --log-level=${LOGLEVEL} \
    app:app

Now I actually hav tried to set the --workers=1 and there is no duplicate anymore. I havn tested fully but it seems at least related.