ex <- data.table::copy(example_quantile)
ex <- ex[!(model == "epiforecasts-EpiNow2" & quantile_level < 0.7)]
as_forecast(ex) |>
score()
This fails because wis can't be computed for one of the models (as there aren't any intervals left). The column wis just doesn't get computed for the EpiNow2 forecasts (but for all other forecasts). This results in an error when binding the results together.
The following, however, "works" (i.e. produces an output):
ex <- ex[!(quantile_level < 0.7)]
as_forecast(ex) |>
score()
I just encountered this problem when running an evaluation. Given that we currently continue execution even when something errors, my suggested fix would be to add fill=TRUE for rbindlist within score
Related (and to be addressed in a separate issue):
some metrics produce NA when they error, others just error. We should probably make that consistent.
Consider the following case:
This fails because
wiscan't be computed for one of the models (as there aren't any intervals left). The columnwisjust doesn't get computed for the EpiNow2 forecasts (but for all other forecasts). This results in an error when binding the results together.The following, however, "works" (i.e. produces an output):
I just encountered this problem when running an evaluation. Given that we currently continue execution even when something errors, my suggested fix would be to add
fill=TRUEforrbindlistwithinscoreRelated (and to be addressed in a separate issue): some metrics produce
NAwhen they error, others just error. We should probably make that consistent.