search

dabapps / django-log-request-id

Django middleware and log filter to attach a unique ID to every log message generated as part of a request
BSD 2-Clause "Simplified" License
371 stars 64 forks source link

Log request id set to [none] in thread pool executor. #63

Closed prashantidealittechno closed 2 years ago

prashantidealittechno commented 2 years ago

In the thread pull executor, I am calling multiple 3rd party APIs in parallel but in the log, it set the log request id none.

ERROR    [none] [2022-01-19 12:57:16,200] weather.apis:  for attempt : 1
ERROR    [none] [2022-01-19 12:57:16,220] weather.apis:  for attempt : 2
ERROR    [none] [2022-01-19 12:57:16,239] weather.apis:  for attempt : 3
ERROR    [none] [2022-01-19 12:57:16,239] weather.apis: Vendor API failed.

Link for thread pool executor: https://docs.python.org/3/library/concurrent.futures.html

j4mie commented 2 years ago

Hi,

The request ID is stored on a thread local so this is not going to work. A thread pool reuses threads, so copying across the request ID automatically isn't likely to be a good idea either (even if it were possible).

If you need to correlate your background-work threads with the request-handling thread, I'd suggest explicitly passing your request ID along to the functions that run in your threads and then logging the ID manually.

I'm open to suggestions on other ways to solve this but I'll close the issue for now as it's not something that is supported currently.