Closed jdberson closed 7 months ago
Thanks for the issue, @jdberson!
Does https://github.com/tidymodels/finetune/pull/100 do the trick for you? You can install it with devtools::install_github("tidymodels/finetune#100")
.
Hi @simonpcouch
Thanks very much for looking into this. Unfortunately I'm still getting an error message when using parallel computing.
I've included a reprex in case it's useful.
library(tidymodels)
library(spatialsample)
library(finetune)
library(sf)
#> Linking to GEOS 3.11.2, GDAL 3.7.2, PROJ 9.3.0; sf_use_s2() is TRUE
# Data
data("ames", package = "modeldata")
# Convert to sf object for spatial resampling
ames_sf <- sf::st_as_sf(
x = ames[1:200, ],
coords = c("Longitude", "Latitude"),
crs = 4326
)
# Spatial resampling using the spatialsample package
set.seed(123)
spatial_block_folds <- spatial_block_cv(ames_sf, v = 5)
# Workflow
bart_spec <-
parsnip::bart(trees = tune()) |>
set_mode("regression") |>
set_engine("dbarts")
bart_rec <-
recipe(Sale_Price ~ Year_Built + Bldg_Type + Gr_Liv_Area,
data = ames
)
bart_wflow <-
workflow() |>
add_model(bart_spec) |>
add_recipe(bart_rec)
# Tuning using tune_race_anova()
cores <- parallel::detectCores(logical = FALSE)
cl <- parallel::makePSOCKcluster(cores)
doParallel::registerDoParallel(cl)
tune_race_with_control <-
bart_wflow |>
tune_race_anova(
resamples = spatial_block_folds,
control = control_race(pkgs = "sf")
)
#> Warning: All models failed. Run `show_notes(.Last.tune.result)` for more
#> information.
#> Error in `test_parameters_gls()`:
#> ! There were no valid metrics for the ANOVA model.
#> Backtrace:
#> ▆
#> 1. ├─finetune::tune_race_anova(...)
#> 2. └─finetune:::tune_race_anova.workflow(...)
#> 3. └─finetune:::tune_race_anova_workflow(...)
#> 4. └─finetune:::test_parameters_gls(res, control$alpha, opt_metric_time)
#> 5. └─cli::cli_abort("There were no valid metrics for the ANOVA model.")
#> 6. └─rlang::abort(...)
# Show more information
show_notes(.Last.tune.result)
#> unique notes:
#> ─────────────────────────────────────────────────────
#> Error in `vec_size()`:
#> ! `x` must be a vector, not a <sfc_POINT/sfc> object.
Created on 2024-01-22 with reprex v2.0.2
Ah, I see. #74 should do the trick. :)
The problem
I'm having trouble using the tune_race_anova() function with a spatial_block_cv object from the spatialsample package whilst using parallel computing. I receive the error “There were no valid metrics for the ANOVA model.” and the show_notes(.Last.tune.result) gives Error in
FUN()
: !x
must be a vector, not a <sfc_POINT/sfc> object. The error does not occur if I don't use parallel computing.I get a similar error when using the tune_grid() function from the tune package. However, if I specify control = control_grid(pkgs = "sf"), the tune_grid() function will work with the spatial_block_cv object and parallel computing.
I think the issue is that specifying control = control_race(pkgs = "sf") in the tune_race_anova() function is not being passed to control$pkgs (line 232 in the function code). I can get the tune_race_anova() function to work with parallel computing and a spatial_block_cv object if I modify the function by including “sf” in the list of packages passed to control$pkgs.
The reprex shows tuning using the tune_grid() function, with and without the control = control_grid(pkgs = "sf") argument to show the error and how it is addressed; as well as tuning using the tune_race_anova() function with and without the control = control_race(pkgs = "sf") to show that the error remains.
I did have some success using the workaround suggested for #39 which I have included at the end of the reprex.
Thanks for your help!
Reproducible example
Created on 2023-05-30 with reprex v2.0.2
Session info
``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.2.3 (2023-03-15 ucrt) #> os Windows 10 x64 (build 19042) #> system x86_64, mingw32 #> ui RTerm #> language (EN) #> collate English_Australia.utf8 #> ctype English_Australia.utf8 #> tz Australia/Perth #> date 2023-05-30 #> pandoc 2.19.2 @ C:/program files/rstudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> backports 1.4.1 2021-12-13 [2] CRAN (R 4.2.0) #> boot 1.3-28.1 2022-11-22 [2] CRAN (R 4.2.3) #> broom * 1.0.4 2023-03-11 [2] CRAN (R 4.2.3) #> cachem 1.0.8 2023-05-01 [1] CRAN (R 4.2.3) #> class 7.3-21 2023-01-23 [2] CRAN (R 4.2.3) #> classInt 0.4-9 2023-02-28 [1] CRAN (R 4.2.2) #> cli 3.6.1 2023-03-23 [1] CRAN (R 4.2.3) #> codetools 0.2-19 2023-02-01 [2] CRAN (R 4.2.3) #> colorspace 2.1-0 2023-01-23 [2] CRAN (R 4.2.3) #> conflicted 1.2.0 2023-02-01 [2] CRAN (R 4.2.3) #> data.table 1.14.8 2023-02-17 [2] CRAN (R 4.2.3) #> dbarts 0.9-23 2023-01-23 [1] CRAN (R 4.2.3) #> DBI 1.1.3 2022-06-18 [2] CRAN (R 4.2.3) #> dials * 1.2.0 2023-04-03 [2] CRAN (R 4.2.3) #> DiceDesign 1.9 2021-02-13 [2] CRAN (R 4.2.3) #> digest 0.6.31 2022-12-11 [2] CRAN (R 4.2.3) #> doParallel 1.0.17 2022-02-07 [1] CRAN (R 4.2.3) #> dplyr * 1.1.2 2023-04-20 [1] CRAN (R 4.2.3) #> e1071 1.7-13 2023-02-01 [1] CRAN (R 4.2.2) #> evaluate 0.21 2023-05-05 [1] CRAN (R 4.2.3) #> fansi 1.0.4 2023-01-22 [1] CRAN (R 4.2.2) #> fastmap 1.1.1 2023-02-24 [2] CRAN (R 4.2.3) #> finetune * 1.1.0 2023-04-19 [1] CRAN (R 4.2.3) #> foreach 1.5.2 2022-02-02 [2] CRAN (R 4.2.3) #> fs 1.6.2 2023-04-25 [1] CRAN (R 4.2.3) #> furrr 0.3.1 2022-08-15 [2] CRAN (R 4.2.3) #> future 1.32.0 2023-03-07 [2] CRAN (R 4.2.3) #> future.apply 1.10.0 2022-11-05 [2] CRAN (R 4.2.3) #> generics 0.1.3 2022-07-05 [2] CRAN (R 4.2.3) #> ggplot2 * 3.4.2 2023-04-03 [1] CRAN (R 4.2.3) #> globals 0.16.2 2022-11-21 [2] CRAN (R 4.2.2) #> glue 1.6.2 2022-02-24 [2] CRAN (R 4.2.3) #> gower 1.0.1 2022-12-22 [2] CRAN (R 4.2.2) #> GPfit 1.0-8 2019-02-08 [2] CRAN (R 4.2.3) #> gtable 0.3.3 2023-03-21 [2] CRAN (R 4.2.3) #> hardhat 1.3.0 2023-03-30 [2] CRAN (R 4.2.3) #> htmltools 0.5.5 2023-03-23 [2] CRAN (R 4.2.3) #> infer * 1.0.4 2022-12-02 [2] CRAN (R 4.2.3) #> ipred 0.9-14 2023-03-09 [2] CRAN (R 4.2.3) #> iterators 1.0.14 2022-02-05 [2] CRAN (R 4.2.3) #> KernSmooth 2.23-20 2021-05-03 [2] CRAN (R 4.2.3) #> knitr 1.42 2023-01-25 [2] CRAN (R 4.2.3) #> lattice 0.20-45 2021-09-22 [2] CRAN (R 4.2.3) #> lava 1.7.2.1 2023-02-27 [2] CRAN (R 4.2.3) #> lhs 1.1.6 2022-12-17 [2] CRAN (R 4.2.3) #> lifecycle 1.0.3 2022-10-07 [2] CRAN (R 4.2.3) #> listenv 0.9.0 2022-12-16 [2] CRAN (R 4.2.3) #> lme4 1.1-33 2023-04-25 [1] CRAN (R 4.2.3) #> lubridate 1.9.2 2023-02-10 [1] CRAN (R 4.2.2) #> magrittr 2.0.3 2022-03-30 [2] CRAN (R 4.2.3) #> MASS 7.3-58.2 2023-01-23 [2] CRAN (R 4.2.3) #> Matrix 1.5-3 2022-11-11 [1] CRAN (R 4.2.2) #> memoise 2.0.1 2021-11-26 [2] CRAN (R 4.2.3) #> minqa 1.2.5 2022-10-19 [2] CRAN (R 4.2.3) #> modeldata * 1.1.0 2023-01-25 [2] CRAN (R 4.2.3) #> munsell 0.5.0 2018-06-12 [2] CRAN (R 4.2.3) #> nlme 3.1-162 2023-01-31 [2] CRAN (R 4.2.3) #> nloptr 2.0.3 2022-05-26 [2] CRAN (R 4.2.3) #> nnet 7.3-18 2022-09-28 [2] CRAN (R 4.2.3) #> parallelly 1.35.0 2023-03-23 [2] CRAN (R 4.2.3) #> parsnip * 1.1.0 2023-04-12 [2] CRAN (R 4.2.3) #> pillar 1.9.0 2023-03-22 [2] CRAN (R 4.2.3) #> pkgconfig 2.0.3 2019-09-22 [2] CRAN (R 4.2.3) #> prodlim 2023.03.31 2023-04-02 [2] CRAN (R 4.2.3) #> proxy 0.4-27 2022-06-09 [2] CRAN (R 4.2.3) #> purrr * 1.0.1 2023-01-10 [1] CRAN (R 4.2.2) #> R6 2.5.1 2021-08-19 [2] CRAN (R 4.2.3) #> Rcpp 1.0.10 2023-01-22 [1] CRAN (R 4.2.2) #> recipes * 1.0.6 2023-04-25 [2] CRAN (R 4.2.3) #> reprex 2.0.2 2022-08-17 [1] CRAN (R 4.2.3) #> rlang 1.1.1 2023-04-28 [1] CRAN (R 4.2.3) #> rmarkdown 2.21 2023-03-26 [2] CRAN (R 4.2.3) #> rpart 4.1.19 2022-10-21 [2] CRAN (R 4.2.3) #> rsample * 1.1.1 2022-12-07 [2] CRAN (R 4.2.3) #> rstudioapi 0.14 2022-08-22 [2] CRAN (R 4.2.3) #> s2 1.1.4 2023-05-17 [1] CRAN (R 4.2.3) #> scales * 1.2.1 2022-08-20 [2] CRAN (R 4.2.3) #> sessioninfo 1.2.2 2021-12-06 [2] CRAN (R 4.2.3) #> sf * 1.0-12 2023-03-19 [1] CRAN (R 4.2.3) #> spatialsample * 0.4.0 2023-05-17 [1] CRAN (R 4.2.3) #> survival 3.5-3 2023-02-12 [2] CRAN (R 4.2.3) #> tibble * 3.2.1 2023-03-20 [1] CRAN (R 4.2.3) #> tidymodels * 1.1.0 2023-05-01 [1] CRAN (R 4.2.3) #> tidyr * 1.3.0 2023-01-24 [1] CRAN (R 4.2.2) #> tidyselect 1.2.0 2022-10-10 [2] CRAN (R 4.2.3) #> timechange 0.2.0 2023-01-11 [1] CRAN (R 4.2.2) #> timeDate 4022.108 2023-01-07 [2] CRAN (R 4.2.3) #> tune * 1.1.1 2023-04-11 [2] CRAN (R 4.2.3) #> units 0.8-2 2023-04-27 [1] CRAN (R 4.2.3) #> utf8 1.2.3 2023-01-31 [1] CRAN (R 4.2.2) #> vctrs 0.6.2 2023-04-19 [1] CRAN (R 4.2.3) #> withr 2.5.0 2022-03-03 [2] CRAN (R 4.2.3) #> wk 0.7.3 2023-05-06 [1] CRAN (R 4.2.3) #> workflows * 1.1.3 2023-02-22 [2] CRAN (R 4.2.3) #> workflowsets * 1.0.1 2023-04-06 [2] CRAN (R 4.2.3) #> xfun 0.39 2023-04-20 [1] CRAN (R 4.2.3) #> yaml 2.3.7 2023-01-23 [2] CRAN (R 4.2.3) #> yardstick * 1.2.0 2023-04-21 [2] CRAN (R 4.2.3) #> #> [1] C:/Users/00055815/AppData/Local/R/win-library/4.2 #> [2] C:/Program Files/R/R-4.2.3/library #> #> ────────────────────────────────────────────────────────────────────────────── ```