search

zachmayer / caretEnsemble

caret models all the way down :turtle:
http://zachmayer.github.io/caretEnsemble/
Other
226 stars 74 forks source link

Auto import for the predict method #14

Closed jknowles closed 2 months ago

jknowles commented 10 years ago

If you ensemble a series of models using external packages (e.g. mda or glmnet) and then clear your workspace and R session and load the caretEnsemble object and attempt to use the predict method, it fails.

This is because the predict method for each individual model type is not necessarily in the namespace. The predict.caretEnsemble should be able to identify and import the predict methods necessary to generate predictions for each model type to be ensembled.

zachmayer commented 9 years ago

Here's a script that reproduces the error:

library(glmnet)
library(mda)
library(rpart)
x <- as.matrix(iris[,-c(4:5)])
m1 <- glmnet(x, iris[,4])
m2 <- mda(Species~., iris)
m3 <- rpart(Species~., iris)
save(m1, m2, m3, x, file='~/Desktop/m.rda')

In a new session:

load('~/Desktop/m.rda')
p1 <- predict(m1, x)
p2 <- predict(m2, iris)
p3 <- predict(m3, iris)

glmnet and mda fail, rpart works. Honestly, this seems to me to be a bug in the glment and mda packages. Saved S3 objects should know how to find their methods....

zachmayer commented 9 years ago

It seems like the problem is the child classes (elnet and mda) that inherit from parent classes (glmnet, fda). Neither package defines S3 methods for the child classes,

zachmayer commented 9 years ago

I think the problem would be fixed if glmnet included S3method("predict", "elnet") and mda included S3method("predict", "mda") in their namespace.

I can't find an issue tracker for either package, but they're both maintained by Trevor Hastie (A fairly well-known statistician). Do you think it's worth emailing him a feature request?

zachmayer commented 9 years ago

Hmmm, mda includes S3method("predict", "mda") and S3method("predict", "fda") in it's namespace, so that's not the issue.

However, mda does NOT export it's predict function. Maybe adding predict.mda to mda's exports and S3method("predict", "elnet") to glmnet would fix it.

jknowles commented 9 years ago

@zachmayer , we should definitely e-mail him. I think these are very useful prediction functions and I have at least 1 major bug in a production system that I have to write a hacked up workaround as a result of this. Would be very beneficial to others I think!

zachmayer commented 9 years ago

I emailed Trevor Hastie and he said he'll fix the issue if he can find a mechanism for doing so. What other packages trigger this error?

jknowles commented 9 years ago

I'll run a check. I have run into it on a few regression models as well -- I should be able to get a report some time this week.

zachmayer commented 9 years ago

I'll checking with Trevor Hastie sometime early next year and see if he thinks this is fixable.

jknowles commented 9 years ago

I'll try to build a test case for this by looping through caret methods with my datasets.

zachmayer commented 9 years ago

Sounds good!

zachmayer commented 9 years ago

I pinged Prof. Hastie about this, and he said it's on his list, but he's not sure when he'll get to it.

Perhaps some industrious person could submit a patch to him. I'm note sure where the mda and glment source code lives.

zachmayer commented 2 months ago

predict should be using s3 classes now