Tuning MLPs with h2o

[walkerjameschris]

The problem

I am having trouble tuning an MLP using tune when the engine is set to "h2o". I am creating a recipe, applying it to a workflow, and tuning it over a grid of parameters. This tuning procedure works for other engines such as "nnet". The error appears to be related to performance computation:

#> x Bootstrap1: internal: Error in fn(...): unused arguments (metrics_info = list(c("rmse", "rsq")...

Reproducible example

library(tidymodels)
library(tidyr)
library(ggplot2)
library(agua)

nn_rec <-
  recipes::recipe(
    price ~ .,
    data = diamonds
  ) |>
  recipes::step_select(
    recipes::all_numeric()
  )

nn_grid <-
  tidyr::expand_grid(
    hidden_units = c(5, 10)
  )

nn_wflow <-
  parsnip::mlp(
    mode = "regression",
    engine = "h2o",
    hidden_units = tune::tune()
  ) |>
  workflows::workflow(
    preprocessor = nn_rec,
    spec = _
  )

rsample::bootstraps(
  data = diamonds,
  times = 3
) |>
  withr::with_seed(
    seed = 12345
  ) |>
  tune::tune_grid(
    object = nn_wflow,
    resamples = _,
    grid = nn_grid,
    control = tune::control_grid(verbose = TRUE)
  )
#> x Bootstrap1: internal: Error in fn(...): unused arguments (metrics_info = list(c("rmse", "rsq")...
#> x Bootstrap2: internal: Error in fn(...): unused arguments (metrics_info = list(c("rmse", "rsq")...
#> x Bootstrap3: internal: Error in fn(...): unused arguments (metrics_info = list(c("rmse", "rsq")...
#> Warning: All models failed. Run `show_notes(.Last.tune.result)` for more
#> information.
#> # Tuning results
#> # Bootstrap sampling 
#> # A tibble: 3 × 4
#>   splits                id         .metrics .notes          
#>   <list>                <chr>      <list>   <list>          
#> 1 <split [53940/19788]> Bootstrap1 <NULL>   <tibble [1 × 3]>
#> 2 <split [53940/19761]> Bootstrap2 <NULL>   <tibble [1 × 3]>
#> 3 <split [53940/19941]> Bootstrap3 <NULL>   <tibble [1 × 3]>
#> 
#> There were issues with some computations:
#> 
#>   - Error(s) x3: Error in fn(...): unused arguments (metrics_info = list(c("rmse",...
#> 
#> Run `show_notes(.Last.tune.result)` for more information.

^{Created on 2024-04-02 with reprex v2.1.0}

[simonpcouch]

Thanks for the issue!

The fixes for this issue should be upstream in agua after https://github.com/tidymodels/agua/pull/52--we need to get a CRAN release of that package out. Related to https://github.com/tidymodels/agua/issues/53; will close this issue in favor of discussion there.

[simonpcouch]

Your solution, for now, would be to install the development version of the package with pak::pak("tidymodels/agua"), restart R, and try again. :)

[walkerjameschris]

Thank you for your quick response! Installing the development version did work.

[github-actions[bot]]

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.