metrics() accept metric options

[jxu]

From https://github.com/tidymodels/tidymodels.org/issues/42

metrics() should be able to pass options to roc_auc(), like event_level, or accept it for all metrics.

I think metric_tweak() can already wrap metric functions in the whole environment? Then metric_set() to calculate a bunch of metrics together?

[EmilHvitfeldt]

As I said in that issue. I don't think it is worth the trouble to have arguments passed through the metrics() function to the underlying metric functions. If you want to do something fancier you can use metric_set() and metric_tweak()

library(yardstick)

metrics(two_class_example, truth, predicted, Class1)
#> # A tibble: 4 × 3
#>   .metric     .estimator .estimate
#>   <chr>       <chr>          <dbl>
#> 1 accuracy    binary         0.838
#> 2 kap         binary         0.675
#> 3 mn_log_loss binary         0.328
#> 4 roc_auc     binary         0.939

# Reproduce `metrics()` results
my_metrics <- metric_set(
  accuracy,
  kap,
  mn_log_loss, 
  roc_auc
)

my_metrics(
  two_class_example, 
  truth = truth, 
  estimate = predicted, 
  Class1
)
#> # A tibble: 4 × 3
#>   .metric     .estimator .estimate
#>   <chr>       <chr>          <dbl>
#> 1 accuracy    binary         0.838
#> 2 kap         binary         0.675
#> 3 mn_log_loss binary         0.328
#> 4 roc_auc     binary         0.939

# using `metric_tweak()` to change event level for `roc_auc()`
my_metrics <- metric_set(
  accuracy,
  kap,
  mn_log_loss, 
  metric_tweak("roc_auc_second", roc_auc,  event_level = "second")
)

my_metrics(
  two_class_example, 
  truth = truth, 
  estimate = predicted, 
  Class1
)
#> # A tibble: 4 × 3
#>   .metric        .estimator .estimate
#>   <chr>          <chr>          <dbl>
#> 1 accuracy       binary        0.838 
#> 2 kap            binary        0.675 
#> 3 mn_log_loss    binary        0.328 
#> 4 roc_auc_second binary        0.0607

^{Created on 2023-10-27 with reprex v2.0.2}

[jxu]

Ok, I think the alternate if metric_set should be pointed out more explicitly in the docs for metrics

[github-actions[bot]]

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.