SimonDedman
commented
6 years ago See this: https://www.r-bloggers.com/error-handling-in-r/ Currently didn't know how to continue when gbm.loop/auto/step crashes. Can use this to fail fast: try quickest highLR values to lowerLR values (divide by 10 step from 0.1?) until it runs, which constrains the possibility space . Would need to get a feel for how BF & LR (&TC?) combine in practice to produce final CV score. If they're hierarchical (LR>BF>TC) then could optimise in order, setting possibility space backwards. TC defined by n.expvars, BF by bfcheck. Optimise CV on LR alone, then optimise BF, then TC.
Done? See Erik Franklin's code.
Trial & error iterative approach to determine the optimal lr for a data set? How? Stop @ whole percentages? Try OPTIM function & see http://r.789695.n4.nabble.com/Optimization-in-R-similar-to-MS-Excel-Solver-td4646759.html Possibly have an option to start with this in the R function. Separate function? Maybe do as separate function then feed into this so the outputs are jkl? or make one uber function but can use all 3 separately. Uberfunction doesn't need the loops? optim: use Method "L-BFGS-B" require(optimx) see: https://stats.stackexchange.com/questions/103495/how-to-find-optimal-values-for-the-tuning-parameters-in-boosting-trees/105653#105653 The caret package in R is tailor made for this. Its train function takes a grid of parameter values and evaluates the performance using various flavors of cross-validation or the bootstrap. The package author has written a book, Applied predictive modeling, which is highly recommended. 5 repeats of 10-fold cross-validation is used throughout the book. For choosing the tree depth, I would first go for subject matter knowledge about the problem, i.e. if you do not expect any interactions - restrict the depth to 1 or go for a flexible parametric model (which is much easier to understand and interpret). That being said, I often find myself tuning the tree depth as subject matter knowledge is often very limited. I think the gbm package tunes the number of trees for fixed values of the tree depth and shrinkage. https://www.youtube.com/watch?v=7Jbb2ItbTC4 see gbm.fixed in BRT_ALL.R - having processed the optimal BRT, might as well just use those details going forward rather than re-running the best one again.