Python 3.6 - Broken Logger handling

[gdude2002]

Billiard: 3.5.0.2 (from PyPi) Python: 3.6.1 Arch Linux

---

Specifically referring to this line, it appears that under Python 3.6, loggers do not have createLock.

[2017-05-12 11:16:53,133: ERROR/PoolWorker-1] Process PoolWorker-1
Traceback (most recent call last):
  File "/home/gdude2002/env/lib/python3.6/site-packages/billiard/process.py", line 294, in _bootstrap
    handler.createLock()
AttributeError: 'Logger' object has no attribute 'createLock'
[2017-05-12 11:16:53,134: WARNING/PoolWorker-1] Process PoolWorker-1:
[2017-05-12 11:16:53,134: WARNING/PoolWorker-1] Traceback (most recent call last):
[2017-05-12 11:16:53,134: WARNING/PoolWorker-1] File "/home/gdude2002/env/lib/python3.6/site-packages/billiard/process.py", line 294, in _bootstrap
    handler.createLock()
[2017-05-12 11:16:53,134: WARNING/PoolWorker-1] AttributeError: 'Logger' object has no attribute 'createLock'
[2017-05-12 11:16:53,182: ERROR/PoolWorker-2] Process PoolWorker-2
Traceback (most recent call last):
  File "/home/gdude2002/env/lib/python3.6/site-packages/billiard/process.py", line 294, in _bootstrap
    handler.createLock()
AttributeError: 'Logger' object has no attribute 'createLock'
[2017-05-12 11:16:53,183: WARNING/PoolWorker-2] Process PoolWorker-2:
[2017-05-12 11:16:53,184: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2017-05-12 11:16:53,183: WARNING/PoolWorker-2] Traceback (most recent call last):
[2017-05-12 11:16:53,188: WARNING/PoolWorker-2] File "/home/gdude2002/env/lib/python3.6/site-packages/billiard/process.py", line 294, in _bootstrap
    handler.createLock()
[2017-05-12 11:16:53,188: WARNING/PoolWorker-2] AttributeError: 'Logger' object has no attribute 'createLock'

However, handlers do. What's going on here?

[gdude2002]

I was able to work around this by never importing logging within any modules my task makes use of. A logger is available from celery.utils.log,get_task_logger(name) instead.

This may be worth documenting somewhere.

[pgcd]

Same happens here (same configuration); unfortunately I cannot really apply the workaround.

UPDATE: After some testing I found that the issue was caused by a library that added logging.NullHandler as a class, rather than an instance. Fortunately it's been updated (just not on PyPI) so my case is solved, but if somebody else is affected I would recomment using the following code (py3.6) to find the culprit and take appropriate steps - it will fail right after printing the name of the logger and the handler.

import logging loggerDict = logging.Logger.manager.loggerDict logger_names = list(loggerDict.keys()) for name in logger_names: if not name or not isinstance(loggerDict[name], logging.PlaceHolder): for handler in logging.getLogger(name).handlers: print(name, handler) handler.createLock()

[auvipy]

I think its fixed on master. could you check the master branch

[rthijs]

I think I ran into the same issue but I'm using pastas. Logged a bug here: https://github.com/pastas/pastas/issues/834

I was not able to find a workaround, never importing logging does not work as simply importing pastas is enough to trigger the error. A workaround would be greatly appreciated.