Closed coforfe closed 7 years ago
Looking at this, I will have train
fit models that can tune over alphas
and nsteps
but use tune = "aic"
for a single combination of those parameters. There doesn't seem to be a way to not tune, so I'll default to AIC (but there will be a single parameter set, so no tuning occurs).
If you can so some testing in the next day or so, that would be a good idea.
Thanks.
Sorry, I have just installed the dev version and I do not see msaenet
there..
Regards, Carlos.
You can get it by sourcing this file then using method = modelInfo
. Thanks
Thanks. I could run it.
I run, several models without any error.
I have tried to replicate a msaenet
example (binomial/classification) and compare it with the equivalent with caret
. For the same dataset both get equivalent results.
Thanks, Carlos.
When I tested, it seemed that the scale
parameter had the largest effect on the results.
Yes, I see the same effect with scale
.
I did this, just for the sake of reproducibility:
With syntetic binary data generated with msaenet
function:
tuneLength = 5
library(caret)
library(msaenet)
cctrl0 <- trainControl(number = 1, classProbs = TRUE,
summaryFunction = twoClassSummary)
dat = msaenet.sim.binomial(
n = 300, p = 500, rho = 0.6,
coef = rep(1, 10), snr = 3, p.train = 0.7,
seed = 1001)
tr_Y <- as.factor(ifelse(dat$y.tr == 0, "Class1", "Class2"))
set.seed(1003)
msa_caret <- train(
x = dat$x.tr,
y = tr_Y,
method = modelInfo,
tuneLength = 5,
trControl = cctrl0,
metric = "ROC"
)
plot(msa_caret)
tuneLength = 5
To check if by scaling the data it gets any improvement:
msa_caret_cent <- train(
x = dat$x.tr,
y = tr_Y,
method = modelInfo,
tuneLength = 5,
trControl = cctrl0,
metric = "ROC",
preProc = c("center", "scale")
)
plot(msa_caret_cent)
This produces an error:
Error in get_types(x) : `x` must have column names
While in this other way:
preProc
and with tuneLength = 5
msa_caret_cent <- train(
x = scale(dat$x.tr),
y = tr_Y,
method = modelInfo,
tuneLength = 5,
trControl = cctrl0,
metric = "ROC",
)
plot(msa_caret_cent)
It works and improves ROC
a little bit.
In both cases, the highest ROC
is achieved with a scale = 2
.
Who doesn't use column names? =]
Just added a check. I'm going to close this one out.
Thanks
@coforfe @topepo -- thanks a lot for adding my method and package to caret. This is awesome! Please just let me know if there is anything I can do to make the API better.
To share some of my experience: SCAD-net and MCP-net usually do better than elastic-net in terms of reducing false positive selections. I use EBIC myself to select the optimal step, but AIC and BIC are also good choices sometimes. scale
can be important, and the adaptive weights transformation is a very interesting direction to explore.
-Nan
Hello,
There is a new algorithm (
regression
) available and specially suitable for p>>n cases:msaenet: Multi-Step Adaptive Elastic-Net
Multi-step adaptive elastic-net (MSAENet) algorithm for feature selection in high-dimensional regressions proposed in Xiao and Xu (2015) (pdf).
Link in CRAN: msaenet
Thanks, Carlos.