Optimal-Learning-Lab / LKT

12 stars 2 forks source link

Single KC model #11

Open ggzheng2196 opened 1 year ago

ggzheng2196 commented 1 year ago

LKT() returns this error when there is only one type of KC in the data and : contrasts can be applied only to factors with 2 or more levels

Example:

Get data

val <- read.delim("...\ds1465_tx_2022_0927_105357\ds1465_tx_All_Data_64_2016_0720_222352.txt", header = TRUE)

Prepare the data for LKT()

val$KC..Default.<-val$Problem.Name val= setDT(val) val$fold<-sample(1:5,length(val$Anon.Student.Id),replace=T) val$CF..Time.<-as.numeric(as.POSIXct(as.character(val$Time),format="%Y-%m-%d %H:%M:%S")) val<-val[order(val$Anon.Student.Id, val$CF..Time.),] val$CF..ansbin.<-ifelse(tolower(val$Outcome)=="correct",1,ifelse(tolower(val$Outcome)=="incorrect",0,-1)) val<-val[val$CF..ansbin==0 | val$CF..ansbin.==1,] val$Duration..sec.<-(val$CF..End.Latency.+val$CF..Review.Latency.+500)/1000 val <- computeSpacingPredictors(val, "KC..Default.")

Fit a single KC LKT model

val.sub = val %>% filter(KC..Default. == "The variance for an observation is the squared difference from the __.") modelob <- LKT( data = val.sub, interc=TRUE, components = c("Anon.Student.Id", "KC..Default."), features = c("intercept", "linesuc$"), fixedpars=c(.9,.5)) # Error in contrasts<-(*tmp*, value = contr.funs[1 + isOF[nn]]) : contrasts can be applied only to factors with 2 or more levels

imrryr commented 1 year ago

Yes, that is how it works currently. It assumes there are multiple levels of all components EXCEPT Anon.Student.Id (that column name exactly), so, when you have 1 level overall (no \$ ) or 1 level per student (with \$ ), you need to use Anon.Student.Id component

Sorry this may seem awkward, in the future it might become cleaner, so I am keeping this issue open to remind me. Thanks for the clear issue report!

Here is the code that worked for me: modelob <- LKT( data = val.sub, interc=TRUE, components = c("Anon.Student.Id", "Anon.Student.Id"), features = c("intercept", "linesuc$"), fixedpars=c(.9,.5))

ggzheng2196 commented 1 year ago

Appreciate the response.

And another question. I have always been confused by the fixedpars argument. How can I find out which parameter a value in that argument is for? Thank you!

imrryr commented 1 year ago

Well the fixed parameters are used for the features that require them in the order listed. So in this example, there is just propdec2, so just 1 fixed parameter. Note how the output lists what parameter is applied to what feature: #> propdec2 KC..Default. 0.9

modelob <- LKT( data = val, interc=TRUE, components = c("Anon.Student.Id", "KC..Default.", "KC..Default.", "KC..Default."), features = c("intercept", "intercept", "propdec2","linefail"), fixedpars=c(.9))

> intercept Anon.Student.Id

> intercept KC..Default.

> propdec2 KC..Default. 0.9

> linefail KC..Default.

> linefailKC..Default.+propdec2KC..Default.+interceptKC..Default.+interceptAnon.Student.Id+1

> McFadden's R2 logistic: 0.303338

> LogLike logistic: -26461.64298933

In this example both logitdec and recency require a parameter, so it is .9 for logitdec and .5 for recency, since they are applied in the order they are supplied.

Recency tracing with logitdec modelob <- LKT( data = val, interc=TRUE, components = c("Anon.Student.Id", "KC..Default.", "KC..Default.", "KC..Default."), features = c("intercept", "intercept", "logitdec","recency"), fixedpars=c(.9,.5))

In this example, ppe requires 4 parameters (the first four, since it is listed first in the list of features) and logitdec gets the last one 0.4443027

modelob <- LKT( data = val, interc=TRUE, components = c("Anon.Student.Id", "KC..Default.", "KC..Default.", "KC..Default."), features = c("intercept", "intercept", "ppe","logitdec"), fixedpars=c(0.3491901,0.2045801,1e-05,0.9734477,0.4443027))

> intercept Anon.Student.Id

> intercept KC..Default.

#> ppe KC..Default. 0.3491901 0.2045801 1e-05 0.9734477

#> logitdec KC..Default. 0.4443027

> logitdecKC..Default.+ppeKC..Default.+interceptKC..Default.+interceptAnon.Student.Id+1

> McFadden's R2 logistic: 0.349833

> LogLike logistic: -24695.58586047

Hope that helps!