Client logging config gets overwritten

[tomuben]

Steps to reproduce

1. Configure logging for client app
2. Use itde from a client application
3. After calling spawn_test_environment / spawn_test_environment_with_test_container log something
4. -> The client log is not printed according to the client app log configuration, but printed to .build_output/jobs/logs/main.log

RCA

1. luigi_log.conf template overwrites the root log channel
2. It would be more correct to configure log channels only used in itde: luigi and luigi-interface, and not root

[tomuben]

Code snippet:

        logging.basicConfig(
            format='%(asctime)s %(levelname)-8s %(message)s',
            level=logging.INFO,
            datefmt='%Y-%m-%d %H:%M:%S')

        self.environment_info, self.clean_up = \
            spawn_test_environment(environment_name="saasd_integration_tests",
                                   database_port_forward=self.database_port,
                                   bucketfs_port_forward=bucketfs_port)

         logging.info("This log gets printed to .build_output/jobs/logs/main.log, but not to stdout as expeccted")

[tkilias]

After first investigation, we know that luigi calls logging.config.fileConfig(opts.logging_conf_file, disable_existing_loggers=False) in the setup_logging.py. The fileConfig call will keep all existing loggers, except the root logger which needs to be defined in the config file. This means the config of the root-loggers gets forgotten. We decided to solve this problem using two strategies.

1. We provide a config which disables the logging setup through luigi completely
2. We make a copy of the root-logger and restore it after the luigi task is done. We can't simply keep a reference on the root logger, we probably need to save each of its elements and restore these.

[morazow]

Closed with #296