library(tidymodels)
library(probably)
#>
#> Attaching package: 'probably'
#> The following objects are masked from 'package:base':
#>
#> as.factor, as.ordered
# Previously, these both gave errors of '! Multiclass not supported...yet'
# The estimate function didn't specify all prediction columns:
segment_logistic %>% cal_estimate_logistic(truth = Class, estimate = .pred_poor)
#> Error in `check_level_consistency()`:
#> ! We can't connect the specified prediction columns to some factor levels (good). The selected columns were .pred_poor. Are there more columns to add in the funciton call?
#> Backtrace:
#> ▆
#> 1. ├─segment_logistic %>% ...
#> 2. ├─probably::cal_estimate_logistic(., truth = Class, estimate = .pred_poor)
#> 3. └─probably:::cal_estimate_logistic.data.frame(...)
#> 4. └─probably:::cal_logistic_impl(...)
#> 5. └─probably:::truth_estimate_map(...)
#> 6. └─probably:::check_level_consistency(truth_levels, est_map)
#> 7. └─rlang::abort(msg)
# (Incomplete specification of the columns _does_ work with the plot funcitons tho)
# Added better error for logistic with 3+ classes
hpc_cv %>% cal_estimate_logistic(truth = obs, estimate = c(VF:L))
#> Error in `cal_logistic_impl()`:
#> ! The number of outcome factor levels isn't consistent with the calibration method. The levels were: 'VF', 'F', 'M', 'L'
#> Backtrace:
#> ▆
#> 1. ├─hpc_cv %>% ...
#> 2. ├─probably::cal_estimate_logistic(., truth = obs, estimate = c(VF:L))
#> 3. └─probably:::cal_estimate_logistic.data.frame(., truth = obs, estimate = c(VF:L))
#> 4. └─probably:::cal_logistic_impl(...)
#> 5. └─rlang::abort(msg)
This pull request has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.
Closes #94
Different error messages:
Created on 2023-05-05 with reprex v2.0.2
I've also updated the docs to say when we extend binary methods to multiclass.