mikemahoney218
commented
6 months ago Sorry, I'm realizing I misdiagnosed the issue here -- I think the real problem is that I didn't name the estimate argument (got too used to the signature for numeric metric sets!). I still the error message here could be more clear about what we need to do to select at least one item, though!
yardstick::metric_set(yardstick::precision)(Orange, age, estimate = Tree)
#> Error in `metric_set()`:
#> ! Failed to compute `precision()`.
#> Caused by error:
#> ! `truth` should be a factor, not a a double vector.
#> Backtrace:
#> ▆
#> 1. ├─(yardstick::metric_set(yardstick::precision))(Orange, age, estimate = Tree)
#> 2. │ └─base::mapply(...)
#> 3. │ └─yardstick (local) `<fn>`(dots[[1L]][[1L]], dots[[2L]][[1L]])
#> 4. │ ├─base::tryCatch(...)
#> 5. │ │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
#> 6. │ │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
#> 7. │ │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
#> 8. │ └─rlang::eval_tidy(expr, data = data, env = env)
#> 9. ├─yardstick (local) `<clss_mtr>`(...)
#> 10. └─yardstick:::precision.data.frame(...)
#> 11. └─yardstick::class_metric_summarizer(...)
#> 12. ├─rlang::inject(...)
#> 13. ├─base::withCallingHandlers(...)
#> 14. └─yardstick (local) fn(...)
#> 15. └─yardstick::check_class_metric(...)
#> 16. └─yardstick:::validate_factor_truth_factor_estimate(...)
#> 17. └─cli::cli_abort(...)
#> 18. └─rlang::abort(...)Created on 2024-04-28 with reprex v2.0.2
The problem
For classification metrics, when
truthorestimateis not a factor,metric_set()gives a much less informative error than the classification metric itself.Reproducible example
Created on 2024-04-28 with reprex v2.0.2
Session info
``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.3.0 (2023-04-21) #> os macOS 14.4.1 #> system aarch64, darwin20 #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz America/Chicago #> date 2024-04-28 #> pandoc 3.1.12.3 @ /Applications/Positron.app/Contents/Resources/app/bin/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> class 7.3-22 2023-05-03 [1] CRAN (R 4.3.0) #> cli 3.6.2.9000 2024-04-04 [1] Github (r-lib/cli@bcb5c78) #> codetools 0.2-19 2023-02-01 [1] CRAN (R 4.3.0) #> colorspace 2.1-0 2023-01-23 [1] CRAN (R 4.3.0) #> data.table 1.14.8 2023-02-17 [1] CRAN (R 4.3.0) #> dials 1.2.0 2023-04-03 [1] CRAN (R 4.3.0) #> DiceDesign 1.9 2021-02-13 [1] CRAN (R 4.3.0) #> digest 0.6.33 2023-07-07 [1] CRAN (R 4.3.0) #> dplyr 1.1.4 2023-11-17 [1] CRAN (R 4.3.1) #> evaluate 0.21 2023-05-05 [1] CRAN (R 4.3.0) #> fansi 1.0.6 2023-12-08 [1] CRAN (R 4.3.1) #> fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.0) #> foreach 1.5.2 2022-02-02 [1] CRAN (R 4.3.0) #> fs 1.6.3 2023-07-20 [1] CRAN (R 4.3.0) #> furrr 0.3.1 2022-08-15 [1] CRAN (R 4.3.0) #> future 1.33.0 2023-07-01 [1] CRAN (R 4.3.0) #> future.apply 1.11.0 2023-05-21 [1] CRAN (R 4.3.0) #> generics 0.1.3 2022-07-05 [1] CRAN (R 4.3.0) #> ggplot2 3.4.2 2023-04-03 [1] CRAN (R 4.3.0) #> globals 0.16.2 2022-11-21 [1] CRAN (R 4.3.0) #> glue 1.7.0 2024-01-09 [1] CRAN (R 4.3.1) #> gower 1.0.1 2022-12-22 [1] CRAN (R 4.3.0) #> GPfit 1.0-8 2019-02-08 [1] CRAN (R 4.3.0) #> gtable 0.3.3 2023-03-21 [1] CRAN (R 4.3.0) #> hardhat 1.3.1 2024-02-02 [1] CRAN (R 4.3.1) #> htmltools 0.5.6 2023-08-10 [1] CRAN (R 4.3.0) #> ipred 0.9-14 2023-03-09 [1] CRAN (R 4.3.0) #> iterators 1.0.14 2022-02-05 [1] CRAN (R 4.3.0) #> knitr 1.43 2023-05-25 [1] CRAN (R 4.3.0) #> lattice 0.22-6 2024-03-20 [1] CRAN (R 4.3.1) #> lava 1.7.2.1 2023-02-27 [1] CRAN (R 4.3.0) #> lhs 1.1.6 2022-12-17 [1] CRAN (R 4.3.0) #> lifecycle 1.0.4 2023-11-07 [1] CRAN (R 4.3.1) #> listenv 0.9.0 2022-12-16 [1] CRAN (R 4.3.0) #> lubridate 1.9.2 2023-02-10 [1] CRAN (R 4.3.0) #> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.3.0) #> MASS 7.3-60.0.1 2024-01-13 [1] CRAN (R 4.3.1) #> Matrix 1.6-5 2024-01-11 [1] CRAN (R 4.3.1) #> munsell 0.5.0 2018-06-12 [1] CRAN (R 4.3.0) #> nnet 7.3-19 2023-05-03 [1] CRAN (R 4.3.0) #> parallelly 1.36.0 2023-05-26 [1] CRAN (R 4.3.0) #> parsnip 1.1.0 2023-04-12 [1] CRAN (R 4.3.0) #> pillar 1.9.0 2023-03-22 [1] CRAN (R 4.3.0) #> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.0) #> prodlim 2023.03.31 2023-04-02 [1] CRAN (R 4.3.0) #> purrr 1.0.2 2023-08-10 [1] CRAN (R 4.3.0) #> R.cache 0.16.0 2022-07-21 [1] CRAN (R 4.3.0) #> R.methodsS3 1.8.2 2022-06-13 [1] CRAN (R 4.3.0) #> R.oo 1.25.0 2022-06-12 [1] CRAN (R 4.3.0) #> R.utils 2.12.2 2022-11-11 [1] CRAN (R 4.3.0) #> R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.0) #> Rcpp 1.0.12 2024-01-09 [1] CRAN (R 4.3.1) #> recipes 1.0.6 2023-04-25 [1] CRAN (R 4.3.0) #> reprex 2.0.2 2022-08-17 [1] CRAN (R 4.3.0) #> rlang 1.1.3.9000 2024-04-04 [1] Github (r-lib/rlang@baedaca) #> rmarkdown 2.24 2023-08-14 [1] CRAN (R 4.3.0) #> rpart 4.1.19 2022-10-21 [1] CRAN (R 4.3.0) #> rsample 1.1.1 2022-12-07 [1] CRAN (R 4.3.0) #> scales 1.2.1 2022-08-20 [1] CRAN (R 4.3.0) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.0) #> styler 1.10.1 2023-06-05 [1] CRAN (R 4.3.0) #> survival 3.5-5 2023-03-12 [1] CRAN (R 4.3.0) #> tibble 3.2.1 2023-03-20 [1] CRAN (R 4.3.0) #> tidyr 1.3.0 2023-01-24 [1] CRAN (R 4.3.0) #> tidyselect 1.2.1 2024-03-11 [1] CRAN (R 4.3.1) #> timechange 0.2.0 2023-01-11 [1] CRAN (R 4.3.0) #> timeDate 4022.108 2023-01-07 [1] CRAN (R 4.3.0) #> tune 1.1.1 2023-04-11 [1] CRAN (R 4.3.0) #> utf8 1.2.4 2023-10-22 [1] CRAN (R 4.3.1) #> vctrs 0.6.5 2023-12-01 [1] CRAN (R 4.3.1) #> withr 3.0.0 2024-01-16 [1] CRAN (R 4.3.1) #> workflows 1.1.3 2023-02-22 [1] CRAN (R 4.3.0) #> xfun 0.40 2023-08-09 [1] CRAN (R 4.3.0) #> yaml 2.3.7 2023-01-23 [1] CRAN (R 4.3.0) #> yardstick 1.3.1.9000 2024-04-28 [1] Github (tidymodels/yardstick@be744a3) #> #> [1] /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library #> #> ────────────────────────────────────────────────────────────────────────────── ```