search

cta-observatory / ctapipe

Low-level data processing pipeline software for CTAO or similar arrays of Imaging Atmospheric Cherenkov Telescopes
https://ctapipe.readthedocs.org
BSD 3-Clause "New" or "Revised" License
64 stars 268 forks source link

Make provenance information in log optional #2328

Closed Tobychev closed 1 year ago

Tobychev commented 1 year ago

The inclusion of all the packages in the environment into the provenance information makes it so long it usually completely dominates the line count of any logfile. This change adds a option that defaults to not including provenance in the log (in addition to it being written to a file).

kosack commented 1 year ago

Another option here is simply to never write the provenance to the stdout logfile. I only had added that for debugging before I had added the provenance.log output. But now that there is a provenance log, it's probably redundant to include it at the DEBUG level.

That way, no new option is needed. I don't really see a use case for turning it on again.

What might be useful in the DEBUG output is just to print the configuration (either from tool.get_current_config() or from the provenance info), but not the complete provenance dictionary

Tobychev commented 1 year ago

@kosack @maxnoe I decided for the "remove it" option, with a note in the logs where for where the interested reader might find the provenance.