Closed dilsherdhillon closed 4 years ago
Also running into this issue
The problem is that your outcome is numeric and you are trying to fit a classification model. We expect a factor predictor.
There is a better error message for this case but that should happen in parsnip
:
library(parsnip)
dta <- tibble::tibble(binary = rbinom(1000, 1, 0.7),
feature_1 = rgamma(1000, 2, 1),
feature_2 = rnorm(1000, 10, 20),
feature_3 = rpois(1000, 5))
### set the engine
engine <-
boost_tree() %>%
set_mode("classification") %>%
set_engine("xgboost")
engine %>% fit(binary ~ ., data = dta)
#> Error: For classification models, the outcome should be a factor.
Created on 2020-02-16 by the reprex package (v0.3.0)
We need to find out where it failed prior to this and write a better message.
I believe this has to do with how catch_and_log_fit()
finds the error to report. It is not logging the thing we want it to.
library(tidymodels)
#> ── Attaching packages ──────────────────────────────────────────── tidymodels 0.1.0 ──
#> ✓ broom 0.5.4 ✓ recipes 0.1.9
#> ✓ dials 0.0.4 ✓ rsample 0.0.5
#> ✓ dplyr 0.8.4 ✓ tibble 2.1.3
#> ✓ ggplot2 3.2.1 ✓ tune 0.0.1.9000
#> ✓ infer 0.5.1 ✓ workflows 0.1.0
#> ✓ parsnip 0.0.5 ✓ yardstick 0.0.5.9000
#> ✓ purrr 0.3.3
#> ── Conflicts ─────────────────────────────────────────────── tidymodels_conflicts() ──
#> x purrr::discard() masks scales::discard()
#> x dplyr::filter() masks stats::filter()
#> x dplyr::lag() masks stats::lag()
#> x ggplot2::margin() masks dials::margin()
#> x recipes::step() masks stats::step()
#> x recipes::yj_trans() masks scales::yj_trans()
dta <- tibble::tibble(binary = rbinom(1000, 1, 0.7),
feature_1 = rgamma(1000, 2, 1),
feature_2 = rnorm(1000, 10, 20),
feature_3 = rpois(1000, 5))
xbg_spec <- boost_tree(mode = "classification", learn_rate = tune()) %>%
set_mode("classification") %>%
set_engine("xgboost")
df_resamples <- vfold_cv(dta, v = 3)
wf <- workflow() %>%
add_model(xbg_spec) %>%
add_formula(binary ~ .)
## error we want
tune:::train_formula(df_resamples$splits[[1]], wf) %>%
tune:::train_model(grid = tibble(learn_rate = 0.00000862),
control = control_workflow())
#> Error: For classification models, the outcome should be a factor.
## NOT error we want
tune:::catch_and_log_fit(
tune:::train_formula(df_resamples$splits[[1]], wf) %>%
tune:::train_model(grid = tibble(learn_rate = 0.00000862),
control = control_workflow()),
control = control_grid()
)
#> Error: $ operator is invalid for atomic vectors
Created on 2020-02-25 by the reprex package (v0.3.0)
It's this line, after the call to catcher()
. The error message we want is caught by catcher()
and stored in result
but then that next line tries to get FIT INCEPTION that isn't there.
I can't find tests for catch_and_log_fit()
. How likely is it that it's behaving in unexpected ways in more than just this situation? What do we think the best next step is?
@topepo Thank you for catching this!
Experiencing this under a different context. I tried to run a model where I specify one argument of the model with tune()
but a hard coded value for the other argument. This raises the same error. See the reprex:
library(tidymodels)
#> ── Attaching packages ────────────────────────────────────── tidymodels 0.1.0 ──
#> ✔ broom 0.5.6 ✔ recipes 0.1.12
#> ✔ dials 0.0.6 ✔ rsample 0.0.6
#> ✔ dplyr 0.8.5 ✔ tibble 3.0.1
#> ✔ ggplot2 3.3.0 ✔ tune 0.1.0
#> ✔ infer 0.5.1 ✔ workflows 0.1.1
#> ✔ parsnip 0.1.1 ✔ yardstick 0.0.6
#> ✔ purrr 0.3.4
#> ── Conflicts ───────────────────────────────────────── tidymodels_conflicts() ──
#> ✖ purrr::discard() masks scales::discard()
#> ✖ dplyr::filter() masks stats::filter()
#> ✖ dplyr::lag() masks stats::lag()
#> ✖ ggplot2::margin() masks dials::margin()
#> ✖ recipes::step() masks stats::step()
library(mlbench)
data(Ionosphere)
Ionosphere <- Ionosphere %>% select(-V2) %>% mutate(cont = 1:nrow(.)) %>% as_tibble()
iono_rec <-
recipe(Class ~ ., data = Ionosphere) %>%
step_zv(all_predictors()) %>%
step_mutate(V1 = factor(V1), Class = factor(Class)) %>%
step_dummy(V1) %>%
step_range(matches("V1_"))
resample <- bootstraps(Ionosphere, times = 5)
# Define a tune for one argument but specify the other
svm_mod <-
svm_rbf(cost = tune(), rbf_sigma = -0.25) %>%
set_mode("classification") %>%
set_engine("kernlab")
# Errors
tune_grid(svm_mod, iono_rec, resample, grid = 3)
#> x Bootstrap1: model 1/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap1: model 2/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap1: model 3/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap2: model 1/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap2: model 2/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap2: model 3/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap3: model 1/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap3: model 2/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap3: model 3/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap4: model 1/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap4: model 2/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap4: model 3/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap5: model 1/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap5: model 2/3 (predictions): Error: $ operator is invalid for atomic vectors
#> x Bootstrap5: model 3/3 (predictions): Error: $ operator is invalid for atomic vectors
#> Warning: All models failed in tune_grid(). See the `.notes` column.
#> # Bootstrap sampling
#> # A tibble: 5 x 4
#> splits id .metrics .notes
#> <list> <chr> <list> <list>
#> 1 <split [351/120]> Bootstrap1 <NULL> <tibble [3 × 1]>
#> 2 <split [351/126]> Bootstrap2 <NULL> <tibble [3 × 1]>
#> 3 <split [351/132]> Bootstrap3 <NULL> <tibble [3 × 1]>
#> 4 <split [351/136]> Bootstrap4 <NULL> <tibble [3 × 1]>
#> 5 <split [351/124]> Bootstrap5 <NULL> <tibble [3 × 1]>
# Define tune for both arguments
svm_mod <-
svm_rbf(cost = tune(), rbf_sigma = tune()) %>%
set_mode("classification") %>%
set_engine("kernlab")
# Runs fine
tune_grid(svm_mod, iono_rec, resample, grid = 3)
#> # Bootstrap sampling
#> # A tibble: 5 x 4
#> splits id .metrics .notes
#> <list> <chr> <list> <list>
#> 1 <split [351/120]> Bootstrap1 <tibble [6 × 5]> <tibble [0 × 1]>
#> 2 <split [351/126]> Bootstrap2 <tibble [6 × 5]> <tibble [0 × 1]>
#> 3 <split [351/132]> Bootstrap3 <tibble [6 × 5]> <tibble [0 × 1]>
#> 4 <split [351/136]> Bootstrap4 <tibble [6 × 5]> <tibble [0 × 1]>
#> 5 <split [351/124]> Bootstrap5 <tibble [6 × 5]> <tibble [0 × 1]>
Here's the SI:
devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.0.0 (2020-04-24)
#> os Ubuntu 20.04 LTS
#> system x86_64, linux-gnu
#> ui X11
#> language (EN)
#> collate en_US.UTF-8
#> ctype en_US.UTF-8
#> tz Europe/Berlin
#> date 2020-05-26
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date lib source
#> assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.0)
#> backports 1.1.7 2020-05-13 [1] CRAN (R 4.0.0)
#> base64enc 0.1-3 2015-07-28 [1] CRAN (R 4.0.0)
#> bayesplot 1.7.1 2019-12-01 [1] CRAN (R 4.0.0)
#> boot 1.3-25 2020-04-26 [3] CRAN (R 4.0.0)
#> broom * 0.5.6 2020-04-20 [1] CRAN (R 4.0.0)
#> callr 3.4.3 2020-03-28 [1] CRAN (R 4.0.0)
#> class 7.3-17 2020-04-26 [3] CRAN (R 4.0.0)
#> cli 2.0.2 2020-02-28 [1] CRAN (R 4.0.0)
#> codetools 0.2-16 2018-12-24 [3] CRAN (R 4.0.0)
#> colorspace 1.4-1 2019-03-18 [1] CRAN (R 4.0.0)
#> colourpicker 1.0 2017-09-27 [1] CRAN (R 4.0.0)
#> crayon 1.3.4 2017-09-16 [1] CRAN (R 4.0.0)
#> crosstalk 1.1.0.1 2020-03-13 [1] CRAN (R 4.0.0)
#> desc 1.2.0 2018-05-01 [1] CRAN (R 4.0.0)
#> devtools 2.3.0 2020-04-10 [1] CRAN (R 4.0.0)
#> dials * 0.0.6 2020-04-03 [1] CRAN (R 4.0.0)
#> DiceDesign 1.8-1 2019-07-31 [1] CRAN (R 4.0.0)
#> digest 0.6.25 2020-02-23 [1] CRAN (R 4.0.0)
#> dplyr * 0.8.5 2020-03-07 [1] CRAN (R 4.0.0)
#> DT 0.13 2020-03-23 [1] CRAN (R 4.0.0)
#> dygraphs 1.1.1.6 2018-07-11 [1] CRAN (R 4.0.0)
#> ellipsis 0.3.1 2020-05-15 [1] CRAN (R 4.0.0)
#> evaluate 0.14 2019-05-28 [1] CRAN (R 4.0.0)
#> fansi 0.4.1 2020-01-08 [1] CRAN (R 4.0.0)
#> fastmap 1.0.1 2019-10-08 [1] CRAN (R 4.0.0)
#> foreach 1.5.0 2020-03-30 [1] CRAN (R 4.0.0)
#> fs 1.4.1 2020-04-04 [1] CRAN (R 4.0.0)
#> furrr 0.1.0 2018-05-16 [1] CRAN (R 4.0.0)
#> future 1.17.0 2020-04-18 [1] CRAN (R 4.0.0)
#> generics 0.0.2 2018-11-29 [1] CRAN (R 4.0.0)
#> ggplot2 * 3.3.0 2020-03-05 [1] CRAN (R 4.0.0)
#> ggridges 0.5.2 2020-01-12 [1] CRAN (R 4.0.0)
#> globals 0.12.5 2019-12-07 [1] CRAN (R 4.0.0)
#> glue 1.4.1 2020-05-13 [1] CRAN (R 4.0.0)
#> gower 0.2.1 2019-05-14 [1] CRAN (R 4.0.0)
#> GPfit 1.0-8 2019-02-08 [1] CRAN (R 4.0.0)
#> gridExtra 2.3 2017-09-09 [1] CRAN (R 4.0.0)
#> gtable 0.3.0 2019-03-25 [1] CRAN (R 4.0.0)
#> gtools 3.8.2 2020-03-31 [1] CRAN (R 4.0.0)
#> hardhat 0.1.3 2020-05-20 [1] CRAN (R 4.0.0)
#> highr 0.8 2019-03-20 [1] CRAN (R 4.0.0)
#> htmltools 0.4.0 2019-10-04 [1] CRAN (R 4.0.0)
#> htmlwidgets 1.5.1 2019-10-08 [1] CRAN (R 4.0.0)
#> httpuv 1.5.2 2019-09-11 [1] CRAN (R 4.0.0)
#> igraph 1.2.5 2020-03-19 [1] CRAN (R 4.0.0)
#> infer * 0.5.1 2019-11-19 [1] CRAN (R 4.0.0)
#> inline 0.3.15 2018-05-18 [1] CRAN (R 4.0.0)
#> ipred 0.9-9 2019-04-28 [1] CRAN (R 4.0.0)
#> iterators 1.0.12 2019-07-26 [1] CRAN (R 4.0.0)
#> janeaustenr 0.1.5 2017-06-10 [1] CRAN (R 4.0.0)
#> kernlab 0.9-29 2019-11-12 [1] CRAN (R 4.0.0)
#> knitr 1.28 2020-02-06 [1] CRAN (R 4.0.0)
#> later 1.0.0 2019-10-04 [1] CRAN (R 4.0.0)
#> lattice 0.20-41 2020-04-02 [3] CRAN (R 4.0.0)
#> lava 1.6.7 2020-03-05 [1] CRAN (R 4.0.0)
#> lhs 1.0.2 2020-04-13 [1] CRAN (R 4.0.0)
#> lifecycle 0.2.0 2020-03-06 [1] CRAN (R 4.0.0)
#> listenv 0.8.0 2019-12-05 [1] CRAN (R 4.0.0)
#> lme4 1.1-23 2020-04-07 [1] CRAN (R 4.0.0)
#> loo 2.2.0 2019-12-19 [1] CRAN (R 4.0.0)
#> lubridate 1.7.8 2020-04-06 [1] CRAN (R 4.0.0)
#> magrittr 1.5 2014-11-22 [1] CRAN (R 4.0.0)
#> markdown 1.1 2019-08-07 [1] CRAN (R 4.0.0)
#> MASS 7.3-51.6 2020-04-26 [3] CRAN (R 4.0.0)
#> Matrix 1.2-18 2019-11-27 [3] CRAN (R 4.0.0)
#> matrixStats 0.56.0 2020-03-13 [1] CRAN (R 4.0.0)
#> memoise 1.1.0 2017-04-21 [1] CRAN (R 4.0.0)
#> mime 0.9 2020-02-04 [1] CRAN (R 4.0.0)
#> miniUI 0.1.1.1 2018-05-18 [1] CRAN (R 4.0.0)
#> minqa 1.2.4 2014-10-09 [1] CRAN (R 4.0.0)
#> mlbench * 2.1-1 2012-07-10 [1] CRAN (R 4.0.0)
#> munsell 0.5.0 2018-06-12 [1] CRAN (R 4.0.0)
#> nlme 3.1-147 2020-04-13 [3] CRAN (R 4.0.0)
#> nloptr 1.2.2.1 2020-03-11 [1] CRAN (R 4.0.0)
#> nnet 7.3-14 2020-04-26 [3] CRAN (R 4.0.0)
#> parsnip * 0.1.1 2020-05-06 [1] CRAN (R 4.0.0)
#> pillar 1.4.4 2020-05-05 [1] CRAN (R 4.0.0)
#> pkgbuild 1.0.8 2020-05-07 [1] CRAN (R 4.0.0)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.0.0)
#> pkgload 1.0.2 2018-10-29 [1] CRAN (R 4.0.0)
#> plyr 1.8.6 2020-03-03 [1] CRAN (R 4.0.0)
#> prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.0.0)
#> pROC 1.16.2 2020-03-19 [1] CRAN (R 4.0.0)
#> processx 3.4.2 2020-02-09 [1] CRAN (R 4.0.0)
#> prodlim 2019.11.13 2019-11-17 [1] CRAN (R 4.0.0)
#> promises 1.1.0 2019-10-04 [1] CRAN (R 4.0.0)
#> ps 1.3.3 2020-05-08 [1] CRAN (R 4.0.0)
#> purrr * 0.3.4 2020-04-17 [1] CRAN (R 4.0.0)
#> R6 2.4.1 2019-11-12 [1] CRAN (R 4.0.0)
#> Rcpp 1.0.4.6 2020-04-09 [1] CRAN (R 4.0.0)
#> recipes * 0.1.12 2020-05-01 [1] CRAN (R 4.0.0)
#> remotes 2.1.1 2020-02-15 [1] CRAN (R 4.0.0)
#> reshape2 1.4.4 2020-04-09 [1] CRAN (R 4.0.0)
#> rlang 0.4.6 2020-05-02 [1] CRAN (R 4.0.0)
#> rmarkdown 2.1 2020-01-20 [1] CRAN (R 4.0.0)
#> rpart 4.1-15 2019-04-12 [3] CRAN (R 4.0.0)
#> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 4.0.0)
#> rsample * 0.0.6 2020-03-31 [1] CRAN (R 4.0.0)
#> rsconnect 0.8.16 2019-12-13 [1] CRAN (R 4.0.0)
#> rstan 2.19.3 2020-02-11 [1] CRAN (R 4.0.0)
#> rstanarm 2.19.3 2020-02-11 [1] CRAN (R 4.0.0)
#> rstantools 2.0.0 2019-09-15 [1] CRAN (R 4.0.0)
#> rstudioapi 0.11 2020-02-07 [1] CRAN (R 4.0.0)
#> scales * 1.1.1 2020-05-11 [1] CRAN (R 4.0.0)
#> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.0)
#> shiny 1.4.0.2 2020-03-13 [1] CRAN (R 4.0.0)
#> shinyjs 1.1 2020-01-13 [1] CRAN (R 4.0.0)
#> shinystan 2.5.0 2018-05-01 [1] CRAN (R 4.0.0)
#> shinythemes 1.1.2 2018-11-06 [1] CRAN (R 4.0.0)
#> SnowballC 0.7.0 2020-04-01 [1] CRAN (R 4.0.0)
#> StanHeaders 2.19.2 2020-02-11 [1] CRAN (R 4.0.0)
#> statmod 1.4.34 2020-02-17 [1] CRAN (R 4.0.0)
#> stringi 1.4.6 2020-02-17 [1] CRAN (R 4.0.0)
#> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.0.0)
#> survival 3.1-12 2020-04-10 [3] CRAN (R 4.0.0)
#> testthat 2.3.2 2020-03-02 [1] CRAN (R 4.0.0)
#> threejs 0.3.3 2020-01-21 [1] CRAN (R 4.0.0)
#> tibble * 3.0.1 2020-04-20 [1] CRAN (R 4.0.0)
#> tidymodels * 0.1.0 2020-02-16 [1] CRAN (R 4.0.0)
#> tidyposterior 0.0.2 2018-11-15 [1] CRAN (R 4.0.0)
#> tidypredict 0.4.5 2020-02-10 [1] CRAN (R 4.0.0)
#> tidyr 1.1.0 2020-05-20 [1] CRAN (R 4.0.0)
#> tidyselect 1.1.0 2020-05-11 [1] CRAN (R 4.0.0)
#> tidytext 0.2.4 2020-04-17 [1] CRAN (R 4.0.0)
#> timeDate 3043.102 2018-02-21 [1] CRAN (R 4.0.0)
#> tokenizers 0.2.1 2018-03-29 [1] CRAN (R 4.0.0)
#> tune * 0.1.0 2020-04-02 [1] CRAN (R 4.0.0)
#> usethis 1.6.1 2020-04-29 [1] CRAN (R 4.0.0)
#> utf8 1.1.4 2018-05-24 [1] CRAN (R 4.0.0)
#> vctrs 0.3.0 2020-05-11 [1] CRAN (R 4.0.0)
#> withr 2.2.0 2020-04-20 [1] CRAN (R 4.0.0)
#> workflows * 0.1.1 2020-03-17 [1] CRAN (R 4.0.0)
#> xfun 0.14 2020-05-20 [1] CRAN (R 4.0.0)
#> xtable 1.8-4 2019-04-21 [1] CRAN (R 4.0.0)
#> xts 0.12-0 2020-01-19 [1] CRAN (R 4.0.0)
#> yaml 2.2.1 2020-02-01 [1] CRAN (R 4.0.0)
#> yardstick * 0.0.6 2020-03-17 [1] CRAN (R 4.0.0)
#> zoo 1.8-8 2020-05-02 [1] CRAN (R 4.0.0)
#>
#> [1] /usr/local/lib/R/site-library
#> [2] /usr/lib/R/site-library
#> [3] /usr/lib/R/library
It is always better to start a new issue when you have an error (and reference this issue). We are unlikely to look at closed issues.
In this particular case, you are using a negative value for the scale parameter in the kernel function. If you have that > 0, it will run fine.
This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.
Hi Folks! I'm running into trouble with the tuning parameters using the
tune_grid
function while usingtune()
to set hyperparameters in {parsnip}.Created on 2020-01-10 by the reprex package (v0.3.0)