tidymodels / dials

Tools for creating tuning parameter values
https://dials.tidymodels.org/
Other
111 stars 26 forks source link

sample_size = sample_prop() quirk when creating tuning grid #212

Closed py9mrg closed 2 years ago

py9mrg commented 2 years ago

Hello,

When doing some manual tuning exploration I am using grid_regular to form a tuning grid. But I have noticed this curious issue where sample_size = sample_prop() will return an error only in the cases where it is the only tuning parameter. If others are present, it is fine.

library(tidymodels)
#> Registered S3 method overwritten by 'tune':
#>   method                   from   
#>   required_pkgs.model_spec parsnip

# works
grid_regular(
    learn_rate()
)
#> # A tibble: 3 x 1
#>     learn_rate
#>          <dbl>
#> 1 0.0000000001
#> 2 0.00000316  
#> 3 0.1

# doesn't work
grid_regular(
    sample_size = sample_prop()
)
#> Error in parameters(list(x, ...)): argument "x" is missing, with no default

# works
grid_regular(
    learn_rate(),
    sample_size = sample_prop()
)
#> # A tibble: 9 x 2
#>     learn_rate sample_size
#>          <dbl>       <dbl>
#> 1 0.0000000001        0.1 
#> 2 0.00000316          0.1 
#> 3 0.1                 0.1 
#> 4 0.0000000001        0.55
#> 5 0.00000316          0.55
#> 6 0.1                 0.55
#> 7 0.0000000001        1   
#> 8 0.00000316          1   
#> 9 0.1                 1

# works
grid_regular(
    sample_size = sample_prop(),
    learn_rate()
)
#> # A tibble: 9 x 2
#>     learn_rate sample_size
#>          <dbl>       <dbl>
#> 1 0.0000000001        0.1 
#> 2 0.00000316          0.1 
#> 3 0.1                 0.1 
#> 4 0.0000000001        0.55
#> 5 0.00000316          0.55
#> 6 0.1                 0.55
#> 7 0.0000000001        1   
#> 8 0.00000316          1   
#> 9 0.1                 1

# to get this to work I either need to use a manual data.frame or:
xgb_spec <- boost_tree(
    mode = "regression",
    sample_size = tune()
)

grid_regular(
    parameters(xgb_spec)
)
#> # A tibble: 3 x 1
#>   sample_size
#>         <dbl>
#> 1        0.1 
#> 2        0.55
#> 3        1

Created on 2022-03-01 by the reprex package (v2.0.1)

The workaround is not the end of the world, but I think this is probably not intended behaviour.

hfrick commented 2 years ago

thanks for the report @py9mrg ! I also think that this isn't necessarily intended 👀

The trouble is that grid_regular(sample_size = sample_prop()) doesn't have an argument x, only sample_size. You can more easily get to what you want via

library(dials)
#> Loading required package: scales

grid_regular(
  list(sample_size = sample_prop())
)
#> # A tibble: 3 × 1
#>   sample_size
#>         <dbl>
#> 1        0.1 
#> 2        0.55
#> 3        1

Created on 2022-03-08 by the reprex package (v2.0.1)

py9mrg commented 2 years ago

No worries @hfrick . Do you want me to close the issue as it's not a very big deal and your solution works nicely, thanks?

hfrick commented 2 years ago

I'll close it since I'm not sure we want to support a method dispatch for anything in the dots when the first argument is missing. If we do, we can open another issue. Thanks for taking the time to report though!

github-actions[bot] commented 2 years ago

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.