tidymodels / workflows

Modeling Workflows
https://workflows.tidymodels.org/
Other
205 stars 23 forks source link

documentation on the add_tailor(prop) #247

Closed topepo closed 2 weeks ago

topepo commented 2 weeks ago

The documentation has that the prop argument to add_tailor() is

The proportion of the data in [fit.workflow()] that should be held back specifically for estimating the postprocessor.

I don't think that is the case. I added a debug() on fit.numeric_calibration and running this code yielded a data object in that function with 900 rows:

library(tidymodels)
library(tailor)
library(bonsai)

set.seed(816)
sim_tr <- sim_regression(1000)

bst_spec <- boost_tree(trees = 15, min_n = 50) %>%
  set_mode("regression") %>%
  set_engine("lightgbm")

bst_wflow <- workflow(outcome ~ ., bst_spec)

cal_post <-
  tailor() %>%
  adjust_numeric_calibration(method = "isotonic")
bst_cal_wflow <-
  bst_wflow %>%
  add_tailor(cal_post, prop = .1)

bst_cal_fit <- fit(bst_cal_wflow, sim_tr)
simonpcouch commented 2 weeks ago

Will move this to workflows where add_tailor() lives!

simonpcouch commented 2 weeks ago

Can confirm that this proportion is reversed; the analysis and potato set are unintentionally switched.

github-actions[bot] commented 2 days ago

This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.