[QUESTION] How to do logging in a FastApi container, any logging does not appear

Description I have another project that utilizes fast api using gunicorn running uvicorn workers and supervisor to keep the api up. Recently I came across the issue that none of my logs from files that are not the fast api app are coming through. Initially I tried making an adhoc script to see if it works as well as changing the levels of the logging. I only had success if I set the logging to be at the DEBUG level.

I put together another small project to test out if I would run into this problem with a clean slate and I still couldn't get logging working with a standard

import logging

log = logging.getLogger(__name__)
log.setLevel(logging.INFO)
log.info('help!')

Other steps I took was chmod-ing the /var/log/ directory in case it was a permissions issue but I had no luck. Has anyone else ran into this or have recommendations on how they implemented logging?

Additional context For context I put up the testing repo here: https://github.com/PunkDork21/fastapi-git-test Testing it would be like:

docker-compose up -d
docker exec -it git-test_web_1 bash
python3 ./appy.py

The most of the files are similar to what I have in my real project

Assuming the original issue was solved, it will be automatically closed now. But feel free to add more comments or create new issues.

I've struggled with this for the past few days as well, and only just figured it out. The HTTP request info is stored in the uvicorn.access logs. In order to see that information when running uvicorn via gunicorn (lots of unicorns here!), you'll need the following snippet in your main.py
import logging
from fastapi.logger import logger as fastapi_logger

gunicorn_error_logger = logging.getLogger("gunicorn.error")
gunicorn_logger = logging.getLogger("gunicorn")
uvicorn_access_logger = logging.getLogger("uvicorn.access")
uvicorn_access_logger.handlers = gunicorn_error_logger.handlers

fastapi_logger.handlers = gunicorn_error_logger.handlers

if __name__ != "__main__":
    fastapi_logger.setLevel(gunicorn_logger.level)
else:
    fastapi_logger.setLevel(logging.DEBUG)
This will allow the gunicorn.error logger to handle the uvicorn.access logger, thus allowing the HTTP request information to come through. You don't even need to add --access-log - in the gunicorn command (but thank you for the suggestion, @bcb!) Big, huge thanks to @slhck and @bcb for pointing me in this direction. I hope that this helps others!

@jacob-vincent thanks for sharing this. I am struggling getting my log setup working as well and this has got me 90% of the way there. I am facing 1 more issue I was hoping you may be able to shed some light on... Can't seem to get the logs working for tasks that are kicked off via an AsyncIOScheduler()

from fastapi import FastAPI
from models.db import init_db
from routers.portfolio import router as portfolio_router
from routers.balance import router as balance_router
from routers.transaction import router as transaction_router

from idom import component, html
from idom.backend.fastapi import configure
import os
from tasks import manage_portfolio_task, fetch_liquidity_changes
from apscheduler.schedulers.asyncio import AsyncIOScheduler

# Configure logs
import logging
from fastapi.logger import logger as fastapi_logger
gunicorn_error_logger = logging.getLogger("gunicorn.error")
gunicorn_logger = logging.getLogger("gunicorn")
uvicorn_access_logger = logging.getLogger("uvicorn.access")
uvicorn_access_logger.handlers = gunicorn_error_logger.handlers

fastapi_logger.handlers = gunicorn_error_logger.handlers

if __name__ != "__main__":
    fastapi_logger.setLevel(gunicorn_logger.level)
else:
    fastapi_logger.setLevel(logging.DEBUG)

# Configure endpoints
app = FastAPI()
app.include_router(portfolio_router, prefix="/portfolio", tags=["Portfolio"])
app.include_router(balance_router, prefix="/balance", tags=["Balance"])
app.include_router(transaction_router, prefix="/transaction", tags=["Transaction"])

# Setup Scheduler
scheduler = AsyncIOScheduler()
scheduler.add_job(manage_portfolio_task, "interval", seconds=10)
scheduler.add_job(fetch_liquidity_changes, "interval", minutes=5)
scheduler.start()

# Configure DB
@app.on_event("startup")
async def on_startup():
    # Startup db
    await init_db()

# For local dev
if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

Have a separate issue open here as well but think its very closely related to this one https://github.com/tiangolo/uvicorn-gunicorn-fastapi-docker/issues/227

This issue is still pretty popular. It took me a while to figure this out, so in case it helps other folks:

The default setting for Uvicorn access logs is to write them to STDOUT. However, the default setting for Gunicorn access logs is to discard them without writing them anywhere.

On the other hand, the default setting for Gunicorn error logs is to stream to STDERR.

When running Uvicorn as a worker class for Gunicorn, Uvicorn will attach its logs properly to Gunicorn's access and error loggers, but unless you have configured the access log to write somewhere you will never see them. It is easy to be confused by this because the default behaviors for Uvicorn/Gunicorn are opposite so the logs just seem to disappear.

Given all of this if you re-use the handlers from Gunicorn's error logger for your access logs, you will see output, but it will be STDERR output (or wherever the error logger is configured to go). This probably isn't what you want.

Instead configure the Gunicorn access logger location and your uvicorn/app logs should start showing up. If you are trying to get them to show up in your console, use "-" to send them to STDOUT.

My team just switched to hypercorn from gunicorn to get http/2 and ran into a ton of logging issues. Because the documentation isn't great on that project right now and this discussion is still up, I'll put our solution here in case anyone is searching for a solution and comes across this.

Not sure why hypercorn doesn't respect the log config the same way as gunicorn or uvicorn, but it doesn't today, at least when used in conjunction with fastapi. We needed to apply the config in the logging module before other imports to make sure they all inherited the same settings. This worked for us both at the CLI and in docker.

Here are our files and what we did:

Put this somewhere accessible to whichever file is your main.py or equivalent that instantiates your app.

log_config.json

{
  "version": 1,
  "disable_existing_loggers": false,
  "formatters": {
    "standard": {
      "format": "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
    },
    "minimal": {
      "format": "%(message)s"
    }
  },
  "handlers": {
    "console": {
      "level": "INFO",
      "class": "logging.StreamHandler",
      "formatter": "standard",
      "stream": "ext://sys.stdout"
    },
    "hypercorn": {
      "level": "INFO",
      "class": "logging.StreamHandler",
      "formatter": "minimal",
      "stream": "ext://sys.stdout"
    }
  },
  "root": {
    "handlers": [
      "console"
    ],
    "level": "INFO"
  },
  "loggers": {
    "": {
      "handlers": [
        "console"
      ],
      "level": "INFO",
      "propagate": false
    },
    "hypercorn.error": {
      "handlers": [
        "hypercorn"
      ],
      "level": "INFO",
      "propagate": false
    },
    "hypercorn.access": {
      "handlers": [
        "hypercorn"
      ],
      "level": "INFO",
      "propagate": false
    }
  }
}

Here's the most important part

main.py

import json
import logging
import logging.config
import os

# Need to setup loggers before importing other modules that may use loggers
# Use whatever path you need to grab the log_config.json file
with open(os.path.join(os.path.dirname(__file__), "log_config.json")) as f:
    logging.config.dictConfig(json.load(f))

# All other imports go below here

_LOGGER.getLogger(__name__)

To run with Hypercorn

hypercorn src.main:app --bind 0.0.0.0:${PORT} --workers 4 --access-logfile - --error-logfile - --worker-class trio

tiangolo / uvicorn-gunicorn-fastapi-docker

[QUESTION] How to do logging in a FastApi container, any logging does not appear #19