Currently (v6.0-94), only internal MeanSD function is supported in RFE to compute the variability of the (repeated) CV metrics. As long as one assumes (repeated) CV samples coming from a normal distribution, this is perfectly fine. But in case this assumption is not taken, then reporting only the SD is very limiting.
A simple addition would be to allow the user to specify an e.g. "summarySD" function in caret::rfeControl(functions), which can point to MeanSD by default. A very quick and dirty implementation one can find here. This would allow much more flexibility for custom implementations of selectSize RFE functions since one could pass extra information and not only the SD.
Currently (
v6.0-94), only internalMeanSDfunction is supported in RFE to compute the variability of the (repeated) CV metrics. As long as one assumes (repeated) CV samples coming from a normal distribution, this is perfectly fine. But in case this assumption is not taken, then reporting only the SD is very limiting.A simple addition would be to allow the user to specify an e.g. "
summarySD" function incaret::rfeControl(functions), which can point toMeanSDby default. A very quick and dirty implementation one can find here. This would allow much more flexibility for custom implementations ofselectSizeRFE functions since one could pass extra information and not only the SD.