Closed JeremyMcCormick closed 1 year ago
Here is sample logging output using the new scheme:
hpsmc:INFO Config files found: ['/work/hps/hps-mc/scratch/.hpsmc']
hpsmc.job:INFO Reading additional config from: ['/work/hps/hps-mc/scratch/dummy.cfg']
hpsmc.job:INFO Job ID: None
hpsmc.job:INFO Description: HPS MC Job
hpsmc.job:INFO Job configuration:
[HPSMC]
loglevel=INFO
[Job]
dry_run=False
delete_existing=True
enable_file_chaining=True
check_output_files=True
delete_rundir=False
[DummyComponent]
loglevel=DEBUG
hpsmc.job:INFO Loading job script: ../python/jobs/dummy_job.py
hpsmc.job:INFO Job components loaded: ['dummy']
hpsmc.job:INFO No job parameters were specified!
hpsmc.job:INFO Changing to run dir: /work/hps/hps-mc/scratch
hpsmc.job:INFO Executing 'dummy' with command: dummy
================ Component: dummy ================
hpsmc.component.DummyComponent:DEBUG dummy debug
hpsmc.component.DummyComponent:INFO dummy info
hpsmc.component.DummyComponent:WARNING dummy warn
hpsmc.component.DummyComponent:CRITICAL dummy critical
hpsmc.component.DummyComponent:ERROR dummy error
hpsmc.job:INFO Execution of dummy took 0.0058 second(s) with return code: None
hpsmc.job:INFO Job execution took 0.010637 seconds
hpsmc.job:INFO Running cleanup for component: dummy
hpsmc.job:INFO Successfully finished running job: Dummy
The above has global log level (HPSMC) set to INFO with the component at DEBUG.
@tomeichlersmith
Thank you for the detailed suggestions.
I committed all your proposed changes, as they looked good. This should be ready for final review now.
This PR adds component level logging and tweaks how logging and configuration works, in general.
Logging can be configured using the following settings in the .hpsmc file:
The
loglevel
sets the log level and must be valid according to python logging module conventions.The
logfile
is an optional parameter for sending all logging output to a file. (Note that logfile can only be used for the "global" rather than component level configuration.)Individual components can have their log level set using:
...where "DummyComponent" should be replaced by the actual class name of the component.
A bunch of confusing configuration and setup having to do with logging was removed from the Job class.
The .hpsmc configuration is now loaded globally and available as
hpsmc.global_config
. Additional configurations may be supplied using the "-c" argument to the job command line interface.In testing, I found that the Job class could not be used without calling the function to parse command line arguments, so I updated it so that it could be created manually without the need for argument parsing. This mainly involved adding default arguments where there were none before.
There are a number of other small bug fixes and cleanup included in the PR as well including removal of some dead code.
I tested the changes to logging and config with
examples/beam_slic
and everything looks fine.