spsanderson / healthyR.ai

healthyR.ai - AI package for the healthyverse
http://www.spsanderson.com/healthyR.ai/
Other
16 stars 6 forks source link

Function `hai_auto_wflw_metrics()` #227

Closed spsanderson closed 2 years ago

spsanderson commented 2 years ago

Function to automatically collect metrics from a boilerplate function. Look for an attribute of boilerplate

spsanderson commented 2 years ago

Function:

#' Collect Metrics from Boilerplat Workflows
#' 
#' @family Metric_Collection
#' @family Boiler_Plate
#' 
#' @author Steven P. Sanderson II, MPH
#' 
#' @details This function will extract the metrics from the `hai_auto_` boilerplate
#' functions. This function looks for a specific attribute from the `hai_auto_`
#' functions so that it will extract the `tuned_results` from the tuning process
#' if it has indeed been tuned.
#' 
#' @description This function will extract the metrics from the `hai_auto_` boilerplate
#' functions.
#' 
#' @param .data The output of the `hai_auto_` boilerplate function in it's entirety.
#' 
#' @examples
#' library(healthyR.ai)
#' 
#' data <- iris
#' 
#' rec_obj <- hai_knn_data_prepper(data, Species ~ .)
#' 
#' auto_knn <- hai_auto_knn(
#'  .data = data,
#'  .rec_obj = rec_obj,
#'  .best_metric = "f_meas",
#'  .model_type = "classification",
#'  .grid_size = 1,
#'  .num_cores = 1
#'  )
#'  
#'  hai_auto_wflw_metrics(auto_knn)
#' 
#' @return 
#' A tibble
#' 
#' @export
#' 

hai_auto_wflw_metrics <- function(.data){

  input_data <- .data
  atb <- attributes(input_data)
  tuned <- atb$.tune

  # Checks
  if (!atb$function_type == "boilerplate"){
    rlang::abort(
      message = "You need to pass in a list object from an 'hai_auto_' function.",
      use_cli_format = TRUE
    )
  }

  if (!tuned){
    rlang::abort(
      message = "The object you passed was not tuned, so there are no metrics
      for this function to collect.",
      use_cli_format = TRUE
    )
  }

  # Data
  df <- tune::collect_metrics(input_data$tuned_info$tuned_results)

  # Return
  return(df)

}

Examples:

data <- iris

rec_obj <- hai_knn_data_prepper(data, Species ~ .)

auto_knn <- hai_auto_knn(
  .data = data,
  .rec_obj = rec_obj,
  .best_metric = "f_meas",
  .model_type = "classification",
  .grid_size = 30,
  .num_cores = parallel::detectCores() - 1
)

auto_knn_not_tuned <- hai_auto_knn(
  .data = data,
  .rec_obj = rec_obj,
  .best_metric = "f_meas",
  .model_type = "classification",
  .tune = FALSE
)

hai_auto_wflw_metrics(auto_knn)
hai_auto_wflw_metrics(auto_knn_not_tuned)