This PR configures logging in logging_util.py such that:
PUDL logging messages are not hidden
OGE logging messages are formatted similar to PUDL (see below)
All logging messages from the data pipeline (including PUDL and other imports) are directed to a logfile (data/outputs/data_pipeline.log). The logfile could easily be changed to a different filepath.
Here's an example of running the data pipeline with the new logging system:
Note:
Unfortunately, if you just do logger.info(df), the header won't line up well with the data. So I replaced all of the dataframe printouts to be logger.info("\n" + df.to_string()), which will make everything look nice.
grgmiller
commented
1 year ago
This PR configures logging in
logging_util.pysuch that:data/outputs/data_pipeline.log). The logfile could easily be changed to a different filepath.Here's an example of running the data pipeline with the new logging system:
Note: Unfortunately, if you just do
logger.info(df), the header won't line up well with the data. So I replaced all of the dataframe printouts to belogger.info("\n" + df.to_string()), which will make everything look nice.