search

zachmayer / caretEnsemble

caret models all the way down :turtle:
Other
226 stars 75 forks source link

greedy ensemble #225

Open univ12 opened 7 years ago

univ12 commented 7 years ago

Hi, I wonder if greedy ensembling is implemented in this package. The caretEnsemble functions calls caretStack with "method=glm". In the caretStack description it says "Find a good linear combination of several classification or regression models, using either linear regression, elastic net regression, or greedy optimization", but I think it's not there. I think a glm linear combination is fine, just wonder about the description and vignette.

zachmayer commented 7 years ago

It is not. I personally found that using a penalized linear regression (e.g. glmnet) to find the weights gives much better results than a greedy approach. The greedy code was painful to maintain, so I eventually took it out.

If someone writes a greedy linear regression solver that works with the caret API, it will be easy to add greedy forward selection for weights back into the package.

Feel free to make a PR to update the description or vignette, or I will put it on my todo list and get to it sometime in the next year or so.