Standardize group arguments in functions

tidymodels / probably

Tools for post-processing class probability estimates

https://probably.tidymodels.org/

Other

115 stars 15 forks source link

Standardize group arguments in functions #105

Closed EmilHvitfeldt closed 1 year ago

EmilHvitfeldt commented 1 year ago

This PR aims to tackle part of https://github.com/tidymodels/probably/issues/104. Change 1 specifically.

While it might seems strange, I'm sticking with group over .by, as it is consistent with the existing codebase. Once this is done, I will swap everything over to .by as part of Change 2.

[x] add group argument to cal_estimate_*.data.frame() functions
[x] add group argument to cal_plot_.*.data.frame() functions
[x] Handle .config in callestimate*.tune_results()` functions
[x] Handle .config in callplot*.tune_results()` functions

topepo commented 1 year ago

For multinomial outcomes, much (but not all) of the infrastructure make a series of 1:all problems and applies binomial methods.

While the automatic detection of .config works for binomial, it doesn't find it for multinomial:

hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_windowed(truth = obs, VF:L)
#> Error in hpc_cv %>% mutate(.config = sample(letters[1:2], nrow(hpc_cv), : could not find function "%>%"

hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_breaks(truth = obs, VF:L)
#> Error in hpc_cv %>% mutate(.config = sample(letters[1:2], nrow(hpc_cv), : could not find function "%>%"

library(tidymodels)
library(probably)
#> 
#> Attaching package: 'probably'
#> The following objects are masked from 'package:base':
#> 
#>     as.factor, as.ordered


tidymodels_prefer()
theme_set(theme_bw())
options(pillar.advice = FALSE, pillar.min_title_chars = Inf)


hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_windowed(truth = obs, VF:L)


hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_breaks(truth = obs, VF:L)

^{Created on 2023-05-10 with reprex v2.0.2}

(using group = .config does work though).

topepo commented 1 year ago

For multinomial outcomes, much (but not all) of the infrastructure make a series of 1:all problems and applies binomial methods.

While the automatic detection of .config works for binomial, it doesn't find it for multinomial:

hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_windowed(truth = obs, VF:L)
#> Error in hpc_cv %>% mutate(.config = sample(letters[1:2], nrow(hpc_cv), : could not find function "%>%"

hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_breaks(truth = obs, VF:L)
#> Error in hpc_cv %>% mutate(.config = sample(letters[1:2], nrow(hpc_cv), : could not find function "%>%"

library(tidymodels)
library(probably)
#> 
#> Attaching package: 'probably'
#> The following objects are masked from 'package:base':
#> 
#>     as.factor, as.ordered


tidymodels_prefer()
theme_set(theme_bw())
options(pillar.advice = FALSE, pillar.min_title_chars = Inf)


hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_windowed(truth = obs, VF:L)


hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_breaks(truth = obs, VF:L)

^{Created on 2023-05-10 with reprex v2.0.2}

(using group = .config does work though).

topepo commented 1 year ago

For multinomial outcomes, much (but not all) of the infrastructure make a series of 1:all problems and applies binomial methods.

While the automatic detection of .config works for binomial, it doesn't find it for multinomial:

hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_windowed(truth = obs, VF:L)
#> Error in hpc_cv %>% mutate(.config = sample(letters[1:2], nrow(hpc_cv), : could not find function "%>%"

hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_breaks(truth = obs, VF:L)
#> Error in hpc_cv %>% mutate(.config = sample(letters[1:2], nrow(hpc_cv), : could not find function "%>%"

library(tidymodels)
library(probably)
#> 
#> Attaching package: 'probably'
#> The following objects are masked from 'package:base':
#> 
#>     as.factor, as.ordered


tidymodels_prefer()
theme_set(theme_bw())
options(pillar.advice = FALSE, pillar.min_title_chars = Inf)


hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_windowed(truth = obs, VF:L)


hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_breaks(truth = obs, VF:L)

^{Created on 2023-05-10 with reprex v2.0.2}

(using group = .config does work though).

topepo commented 1 year ago

For multinomial outcomes, much (but not all) of the infrastructure make a series of 1:all problems and applies binomial methods.

While the automatic detection of .config works for binomial, it doesn't find it for multinomial:

hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_windowed(truth = obs, VF:L)
#> Error in hpc_cv %>% mutate(.config = sample(letters[1:2], nrow(hpc_cv), : could not find function "%>%"

hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_breaks(truth = obs, VF:L)
#> Error in hpc_cv %>% mutate(.config = sample(letters[1:2], nrow(hpc_cv), : could not find function "%>%"

library(tidymodels)
library(probably)
#> 
#> Attaching package: 'probably'
#> The following objects are masked from 'package:base':
#> 
#>     as.factor, as.ordered


tidymodels_prefer()
theme_set(theme_bw())
options(pillar.advice = FALSE, pillar.min_title_chars = Inf)


hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_windowed(truth = obs, VF:L)


hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_breaks(truth = obs, VF:L)

^{Created on 2023-05-10 with reprex v2.0.2}

(using group = .config does work though).

EmilHvitfeldt commented 1 year ago

That should be fixed now

library(tidymodels)
library(probably)
#> 
#> Attaching package: 'probably'
#> The following objects are masked from 'package:base':
#> 
#>     as.factor, as.ordered

hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_windowed(truth = obs, VF:L)


hpc_cv %>%
  mutate(.config = sample(letters[1:2], nrow(hpc_cv), replace = TRUE)) %>% 
  cal_plot_breaks(truth = obs, VF:L)

^{Created on 2023-05-11 with reprex v2.0.2}

github-actions[bot] commented 1 year ago

This pull request has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.