search

european-modelling-hubs / covid19-forecast-hub-europe

European Covid-19 Forecast Hub.
https://covid19forecasthub.eu
Other
45 stars 94 forks source link

`restrict_weeks` argument in `score_models()` is not used #678

Closed Bisaloo closed 3 years ago

Bisaloo commented 3 years ago

https://github.com/epiforecasts/covid19-forecast-hub-europe/blob/49d65f25dbcbbb7807362695cfa6f1204c03f8c2/code/evaluation/score_models.r#L17

@sbfnk, is this on purpose? Or should it be there with the function described in the following comment?

https://github.com/epiforecasts/covid19-forecast-hub-europe/blob/49d65f25dbcbbb7807362695cfa6f1204c03f8c2/code/evaluation/score_models.r#L12

sbfnk commented 3 years ago

This can go - the forecast reports have the same filter here but I think the csv files can contain all models, including ones that have not been submitted recently: https://github.com/epiforecasts/covid19-forecast-hub-europe/blob/49d65f25dbcbbb7807362695cfa6f1204c03f8c2/code/reports/rmdchunks/ranking-table.Rmd#L41

Also the continuous_weeks column that's created here is used to create the weighted ensemble only from models that have been submitted for X weeks in a row. https://github.com/epiforecasts/covid19-forecast-hub-europe/blob/49d65f25dbcbbb7807362695cfa6f1204c03f8c2/code/ensemble/methods/create-ensemble-relative-skill.R#L65