hturner
commented
5 years ago Yes, deviance() is designed to return the deviance of the fitted model object.
However you can obtain the deviance with new data using AIC.pltree() as you show in your last 3 lines of code, that is correct.
I figured out that function deviance() return the same result when used with or without the argument newdata. Is that correct? Here an example from "beans" data.
` library(PlackettLuce)
example("beans", package = "PlackettLuce") G <- grouped_rankings(R, rep(seq_len(nrow(beans)), 4)) formula <- as.formula("G ~ maxTN") d <- cbind(G, beans)
split the data into a training and test sample
n <- nrow(d) s <- sample(1:n, n*0.7) train <- d[s, ] test <- d[-s, ]
fit the model with training data
mod <- pltree(formula, data = train)
predict estimates on test data based on fitted model
predict(mod, newdata = test)
the AIC of fitted model
AIC(mod)
the AIC predicted using the test data
AIC(mod, newdata = test) #different than value above (prediction is working)
the deviance of fitted model
deviance(mod)
the deviance predicted using the test data
deviance(mod, newdata = test) #same as the value above (prediction not working)
Does deviance, when using a test data for prediction, should be like this?:
AIC <- AIC(mod, newdata = test) df <- attr(logLik(mod), "df") AIC - (2*df) `