epiforecasts / scoringutils

Utilities for Scoring and Assessing Predictions
https://epiforecasts.io/scoringutils/
Other
48 stars 21 forks source link

underprediction/overprediction/etc. fails if all observations are above/below the medians #973

Closed sbfnk closed 3 weeks ago

sbfnk commented 3 weeks ago

Example:

library("scoringutils")
#> scoringutils 2.0.0 introduces major changes. We'd love your feedback!
#> <https://github.com/epiforecasts/scoringutils/issues>. To use the old version,
#> run: `remotes::install_github('epiforecasts/scoringutils@v1.2.2')`
#> This message is displayed once per session.

set.seed(1234)

samples <- 10
value <- 5
example <- data.frame(
  observed = rep(value, 2),
  predicted = rnorm(samples * 2, value, 1),
  sample_id = rep(seq_len(samples),  2),
  time =  rep(c("2019-01-01", "2019-01-02"), each = samples)
)

score(as_forecast_sample(example))
#> Warning: ! Computation for `overprediction` failed. Error: non-numeric argument to
#>   binary operator.
#> Warning: ! Computation for `underprediction` failed. Error: non-numeric argument to
#>   binary operator.
#> Warning: ! Computation for `dispersion` failed. Error: non-numeric argument to binary
#>   operator.
#>          time  bias        dss      crps log_score       mad ae_median
#>        <char> <num>      <num>     <num>     <num>     <num>     <num>
#> 1: 2019-01-01  -0.2 0.05070157 0.3222352  1.107459 1.1004561 0.5555419
#> 2: 2019-01-02  -0.4 0.03859401 0.3025014  1.100513 0.5937145 0.4941011
#>       se_mean
#>         <num>
#> 1: 0.14680960
#> 2: 0.01396432

Created on 2024-11-02 with reprex v2.1.1