jeffwong-nflx
commented
7 years ago I have attempted to write a solution for this in myfunc where I rely on sparse.model.matrix
require(Matrix)
require(caret)
myfunc <- function(object, newdata, na.action = na.pass, return_sparse = FALSE, ...) {
if(is.null(newdata)) stop("newdata must be supplied")
if(!is.data.frame(newdata)) newdata <- as.data.frame(newdata)
if(!all(object$vars %in% names(newdata))) stop(
paste("Variable(s)",
paste("'", object$vars[object$vars %in% names(newdata)],
"'", sep = "",
collapse = ", "),
"are not in newdata"))
Terms <- object$terms
Terms <- delete.response(Terms)
if(!object$fullRank) {
oldContr <- options("contrasts")$contrasts
newContr <- oldContr
newContr["unordered"] <- "contr.ltfr"
options(contrasts = newContr)
on.exit({
options(contrasts = oldContr)
})
}
m <- model.frame(Terms, newdata, na.action = na.action, xlev = object$lvls)
if (return_sparse) {
x = sparse.model.matrix(Terms, m)
} else {
x = model.matrix(Terms, m)
}
if(object$levelsOnly) {
for(i in object$facVars) {
for(j in object$lvls[[i]]) {
from_text <- paste0(i, j)
colnames(x) <- gsub(from_text, j, colnames(x), fixed = TRUE)
}
}
}
if(!is.null(object$sep) & !object$levelsOnly) {
for(i in object$facVars[order(-nchar(object$facVars))]) {
for(j in object$lvls[[i]]) {
from_text <- paste0(i, j)
to_text <- paste(i, j, sep = object$sep)
colnames(x) <- gsub(from_text, to_text, colnames(x), fixed = TRUE)
}
}
}
x[, colnames(x) != "(Intercept)", drop = FALSE]
}
n = 1000
X = data.frame(a = sample(c('a', 'b', 'c'), n, replace = T),
b = sample(c('d', 'e', 'f'), n, replace = T),
x = 1:n)
X$a = as.factor(X$a)
X$b = as.factor(X$b)
sparse.model.matrix(~ a, X)
foo <- dummyVars(~ ., X, sparse = TRUE, fullRank = FALSE)
bar <- myfunc(foo, X, return_sparse = TRUE, sparse = TRUE)However I get this error
Error in model.spmatrix(t, data, transpose = transpose, drop.unused.levels = drop.unused.levels, :
no slot of name "i" for this object of class "dgeMatrix"I believe it may be related to switching the contrasts option to contr.ltfr, which may not be compatible with sparse.model.matrix? If I comment that block out the code will execute and return a sparse matrix, although it will return one where one level of each factor is dropped (R default)
topepo
Hi, I want to construct a dummyVars matrix in sparse format. Due to my factor variables having a large amount of levels a sparse matrix in the end could really help. Here is a reproducible example
I believe it would be a simple modification here https://github.com/topepo/caret/blob/master/pkg/caret/R/dummyVar.R#L227
This line generates a model.matrix, and we would simply need to allow passing sparse = TRUE and use sparse.model.matrix