askap-vast / vast-post-processing

MIT License
0 stars 0 forks source link

Logging is written to a single file #56

Closed ddobie closed 9 months ago

ddobie commented 9 months ago

Every time the vast_post_processing command is run the logging is simply appended to the existing vast_post_processing.log file.

The log file should be renamed to something like vast_postprocessing{DATETIME}.log.

Possibly related to #42

mubdi commented 9 months ago

So this was done by design, but looking at the mode of operation of the tools here, it likely is the incorrect design.

Currently:

The specific log file is specified (i.e. in the config yaml or the like), and it is a rotating log file in that every run writes to the same place and when the log is too big (i.e., above a certain size in bytes), it rotates the old one to <>.0, and a new logfile is created. This is what you would want to do for a continuously operating service so to prevent the logfiles from growing too large and eating up disk space.

What we actually want:

Since the code is meant to be run as individual runs, the log files shouldn't be rotating, but rather append the UTC ISO datetime of the start of the run to the file name. This would lead to explicit runs having explicit log files.