Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Apache License 2.0
17.71k
stars
2.39k
forks
source link
Fix for applying logging configurations using the a TOML configuration file #3237
Configuring logging using the "logging" session of the TOML configuration file throws an exception because the logging library tries to write to a frozen dictionary. This PR solves the issue by unfreezing the dictionary before passing it to logging.dictConfig.
Motivation and Context
Starting from version 2.8.11, the logging section of the configuration file is loaded into frozen a dictionary which is then passed to the logging.dictConfig when using the TOML parser. However, logging.dictConfig expects a mutable dictionary and tries to overwrites some of its keys, resulting in an exception. In practice that means we cannot use the TOML configuration file to configure logging.
Description
Configuring logging using the "logging" session of the TOML configuration file throws an exception because the
logginglibrary tries to write to a frozen dictionary. This PR solves the issue by unfreezing the dictionary before passing it tologging.dictConfig.Motivation and Context
Starting from version
2.8.11, the logging section of the configuration file is loaded into frozen a dictionary which is then passed to thelogging.dictConfigwhen using the TOML parser. However,logging.dictConfigexpects a mutable dictionary and tries to overwrites some of its keys, resulting in an exception. In practice that means we cannot use the TOML configuration file to configure logging.This problem has been mentioned in the issue https://github.com/spotify/luigi/issues/2879
Have you tested this? If so, how?
Manually tested the changes and added a new test case here to replicate the scenario in which the
masterbranch would throw an error.