prio-data / views_pipeline

VIEWS forecasting pipeline for monthly prediction runs. Includes MLops and QA for all models/ensembles.
Other
3 stars 3 forks source link

Update generel logger #137

Closed Polichinel closed 2 weeks ago

Polichinel commented 2 weeks ago

DELETED OLD PR BC I COULD CHANGE IT TO DEVELOPMENT. All comments taking into account

OLD DESCRIPTION: Updated Xioalongs logger in accordance with Dylan's comment and other considerations. Note that the two functions get_config_log_path and get_common_logs_path will be redundant in a second when the new model_path solution is merged to the main. So see them as placeholders.

The default setting here is to place all logs centrally which is counter to the ADR on logging generated data (009). Here it is stated that:

"## Decision This decision involves implementing a logging system for all generated data and enforcing ensemble model checks. This logging will involve creating a .txt log file in each model-specific folder. The log file will contain the following details:

The name and timestamp of the model artifact that produced the data.
The timestamp of when the data was generated.
Possibly the data stamp of when the raw data used was fetched from VIEWS.
The deployment status of the single model.
"

I don't really know what I think here. I.e. should the central logging be a default that can be overridden if we want to store some logs closer to the thing they pertain to? Or should we force all logs to be centrally stored? I would really like you input here. and we might need to update ADR 009 and 015 accordingly