search

philchalmers / mirt

Multidimensional item response theory
https://philchalmers.github.io/mirt/
201 stars 75 forks source link

Enable custom plot labels on empirical_ES #193

Closed awmeade closed 4 years ago

philchalmers commented 4 years ago

Hi Adam,

Could you be a little more specific regarding what you're looking for here? Thanks.

awmeade commented 4 years ago

Hi Phil!

I received an email request to be able to supply custom labels to the plots generated by empirical_ES. Specifically the ability to change the main title, but the ability to supply labels for 'foc' and 'ref' would be useful too. I looked at the code and "main" on the plot is currently hard-coded using an if statement. I was planning on working on this rather than bug you with it (if I can ever manage to sync my fork from 2 years ago!). Hope you are well!

Adam

On Wed, Sep 9, 2020 at 11:39 AM Phil Chalmers notifications@github.com wrote:

Hi Adam,

Could you be a little more specific regarding what you're looking for here? Thanks.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/philchalmers/mirt/issues/193#issuecomment-689645068, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACA5DEJ6NK5J42VLJ57VSDTSE6OUVANCNFSM4RCRUVRQ .

philchalmers commented 4 years ago

Hi Adam,

I get what feels like an infinite number of requests for 'tweaks' to mirt's default plots, and I often just reply by saying mirt uses lattice as the graphics driver, which generates objects that can be modified directly. Hence, even if users aren't familiar with packages like gridExtra for modifying the grid objects via functions they can always manually search for the components they want changed and modifying them directly:

library(mirt)

#no DIF
set.seed(12345)
a <- matrix(abs(rnorm(15,1,.3)), ncol=1)
d <- matrix(rnorm(15,0,.7),ncol=1)
temtype <- rep('2PL', nrow(a))
N <- 1000
dataset1 <- simdata(a, d, N, itemtype)
dataset2 <- simdata(a, d, N, itemtype, mu = .1, sigma = matrix(1.5))
dat <- rbind(dataset1, dataset2)
group <- c(rep('Ref', N), rep('Focal', N))
mod <- multipleGroup(dat, 1, group = group,
    invariance = c(colnames(dat)[1:5], 'free_means', 'free_var'))

out1 <- empirical_ES(mod, plot=TRUE)
out1
str(out1)
out1$main <- "My New Title"
out1

out2 <- empirical_ES(mod, plot=TRUE, DIF=FALSE)
out2
str(out2)
out2$legend$top$args$key$text[[1]] <- c('Focal', 'Reference')
out2
philchalmers commented 4 years ago

Based on the current code though I think I can see the user's main complaint: the original group names in the fitted model are not included in the graphic object. That's a pretty easy fix by just using extract.mirt(mod, 'groupNames') to pull out this information instead. If you want to give that a go and send a pull request then let me know, otherwise I can give it a whirl.

awmeade commented 4 years ago

Hi Phil,

First off, I had no idea you could modify the plot so easily! Thanks for letting me know. Very helpful and I passed that info along to the original guy who asked it.

Second, I've taken a stab at using extract.mirt(mod, 'groupnames') in the empirical_ES function and have realized that the extract.mirt function always returns the group names in alphabetical order. I was trying something like: grps <- extract.mirt(mod, 'groupnames') plot.df1 <- data.frame(Theta=Theta.focal[,1],ETS=ETS.foc.obs,group=grps[1]) plot.df2 <- data.frame(Theta=Theta.focal[,1],ETS=ETS.ref.obs,group=grps[2])

This seemed to work at first, but in testing I found that grps[1] is always what comes first alphabetically, not whatever group is specified in: group <- c(rep('Ref', N), rep('Focal', N))

Any idea of a workaround for this issue?

Adam

On Wed, Sep 9, 2020 at 11:58 AM Phil Chalmers notifications@github.com wrote:

Based on the current code though I think I can see the user's main complaint: the original group names in the fitted model are not included in the graphic object. That's a pretty easy fix by just using extract.mirt(mod, 'groupNames') to pull out this information instead. If you want to give that a go and send a pull request then let me know, otherwise I can give it a whirl.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/philchalmers/mirt/issues/193#issuecomment-689656538, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACA5DEKVG7CZNNRYG5RWYNDSE6QZ5ANCNFSM4RCRUVRQ .

philchalmers commented 4 years ago

Second, I've taken a stab at using extract.mirt(mod, 'groupnames') in the empirical_ES function and have realized that the extract.mirt function always returns the group names in alphabetical order.

That's not strictly true, it's actually based on how the factor() function works. If group is supplied as a character vector initial then it is passed to factor(), which dictates the group ordering. If you wanted group B, for instance, to be the control group and group A to be the focal, then you need to initially create group as a different ordered factor. See below for an example of this in case it helps in your code:

library(mirt)

#no DIF
set.seed(12345)
a <- matrix(abs(rnorm(15,1,.3)), ncol=1)
d <- matrix(rnorm(15,0,.7),ncol=1)
temtype <- rep('2PL', nrow(a))
N <- 1000
dataset1 <- simdata(a, d, N, itemtype)
dataset2 <- simdata(a, d, N, itemtype, mu = .1, sigma = matrix(1.5))
dat <- rbind(dataset1, dataset2)
group <- c(rep('B', N), rep('A', N))

str(factor(group)) #default levels

mod <- multipleGroup(dat, 1, group = group,
                     invariance = c(colnames(dat)[1:5], 'free_means', 'free_var'))

extract.mirt(mod, 'groupNames')
coef(mod, simplify=TRUE) # factor A comes first in output

fgroup <- factor(group, levels = c('B', 'A'))
str(fgroup) #default levels

mod2 <- multipleGroup(dat, 1, group = fgroup,
                     invariance = c(colnames(dat)[1:5], 'free_means', 'free_var'))

extract.mirt(mod2, 'groupNames')
coef(mod2, simplify=TRUE) # factor B comes first in output
awmeade commented 4 years ago

Interesting, and I see what you are saying.

I'll play around with this a bit more....

Adam

On Wed, Sep 9, 2020 at 12:55 PM Phil Chalmers notifications@github.com wrote:

Second, I've taken a stab at using extract.mirt(mod, 'groupnames') in the empirical_ES function and have realized that the extract.mirt function always returns the group names in alphabetical order.

That's not strictly true, it's actually based on how the factor() function works. If group is supplied as a character vector initial then it is passed to factor(), which dictates the group ordering. If you wanted group B, for instance, to be the control group and group A to be the focal, then you need to initially create group as a different ordered factor. See below for an example of this in case it helps in your code:

library(mirt)

no DIF

set.seed(12345)a <- matrix(abs(rnorm(15,1,.3)), ncol=1)d <- matrix(rnorm(15,0,.7),ncol=1)temtype <- rep('2PL', nrow(a))N <- 1000dataset1 <- simdata(a, d, N, itemtype)dataset2 <- simdata(a, d, N, itemtype, mu = .1, sigma = matrix(1.5))dat <- rbind(dataset1, dataset2)group <- c(rep('B', N), rep('A', N))

str(factor(group)) #default levels mod <- multipleGroup(dat, 1, group = group, invariance = c(colnames(dat)[1:5], 'free_means', 'free_var'))

extract.mirt(mod, 'groupNames') coef(mod, simplify=TRUE) # factor A comes first in output

fgroup <- factor(group, levels = c('B', 'A')) str(fgroup) #default levels mod2 <- multipleGroup(dat, 1, group = fgroup, invariance = c(colnames(dat)[1:5], 'free_means', 'free_var'))

extract.mirt(mod2, 'groupNames') coef(mod2, simplify=TRUE) # factor A comes first in output

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/philchalmers/mirt/issues/193#issuecomment-689689723, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACA5DEK367MG2CICO56W2SLSE6XP3ANCNFSM4RCRUVRQ .

awmeade commented 4 years ago

I'm thinking maybe: grps <- table(extract.mirt(mod, 'group')) grps[1] would be best to get group labels

it appears that extract.mirt('groups') always pulls things in the order in which they appear which may be a lot simpler than pulling 'groupNames' which may or may not reflect the order of the datalabels.

Adam

On Wed, Sep 9, 2020 at 1:57 PM Adam Meade awmeademail@gmail.com wrote:

Interesting, and I see what you are saying.

I'll play around with this a bit more....

Adam

On Wed, Sep 9, 2020 at 12:55 PM Phil Chalmers notifications@github.com wrote:

Second, I've taken a stab at using extract.mirt(mod, 'groupnames') in the empirical_ES function and have realized that the extract.mirt function always returns the group names in alphabetical order.

That's not strictly true, it's actually based on how the factor() function works. If group is supplied as a character vector initial then it is passed to factor(), which dictates the group ordering. If you wanted group B, for instance, to be the control group and group A to be the focal, then you need to initially create group as a different ordered factor. See below for an example of this in case it helps in your code:

library(mirt)

no DIF

set.seed(12345)a <- matrix(abs(rnorm(15,1,.3)), ncol=1)d <- matrix(rnorm(15,0,.7),ncol=1)temtype <- rep('2PL', nrow(a))N <- 1000dataset1 <- simdata(a, d, N, itemtype)dataset2 <- simdata(a, d, N, itemtype, mu = .1, sigma = matrix(1.5))dat <- rbind(dataset1, dataset2)group <- c(rep('B', N), rep('A', N))

str(factor(group)) #default levels mod <- multipleGroup(dat, 1, group = group, invariance = c(colnames(dat)[1:5], 'free_means', 'free_var'))

extract.mirt(mod, 'groupNames') coef(mod, simplify=TRUE) # factor A comes first in output

fgroup <- factor(group, levels = c('B', 'A')) str(fgroup) #default levels mod2 <- multipleGroup(dat, 1, group = fgroup, invariance = c(colnames(dat)[1:5], 'free_means', 'free_var'))

extract.mirt(mod2, 'groupNames') coef(mod2, simplify=TRUE) # factor A comes first in output

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/philchalmers/mirt/issues/193#issuecomment-689689723, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACA5DEK367MG2CICO56W2SLSE6XP3ANCNFSM4RCRUVRQ .

awmeade commented 4 years ago

scratch that.... still working

On Wed, Sep 9, 2020 at 2:56 PM Adam Meade awmeademail@gmail.com wrote:

I'm thinking maybe: grps <- table(extract.mirt(mod, 'group')) grps[1] would be best to get group labels

it appears that extract.mirt('groups') always pulls things in the order in which they appear which may be a lot simpler than pulling 'groupNames' which may or may not reflect the order of the datalabels.

Adam

On Wed, Sep 9, 2020 at 1:57 PM Adam Meade awmeademail@gmail.com wrote:

Interesting, and I see what you are saying.

I'll play around with this a bit more....

Adam

On Wed, Sep 9, 2020 at 12:55 PM Phil Chalmers notifications@github.com wrote:

Second, I've taken a stab at using extract.mirt(mod, 'groupnames') in the empirical_ES function and have realized that the extract.mirt function always returns the group names in alphabetical order.

That's not strictly true, it's actually based on how the factor() function works. If group is supplied as a character vector initial then it is passed to factor(), which dictates the group ordering. If you wanted group B, for instance, to be the control group and group A to be the focal, then you need to initially create group as a different ordered factor. See below for an example of this in case it helps in your code:

library(mirt)

no DIF

set.seed(12345)a <- matrix(abs(rnorm(15,1,.3)), ncol=1)d <- matrix(rnorm(15,0,.7),ncol=1)temtype <- rep('2PL', nrow(a))N <- 1000dataset1 <- simdata(a, d, N, itemtype)dataset2 <- simdata(a, d, N, itemtype, mu = .1, sigma = matrix(1.5))dat <- rbind(dataset1, dataset2)group <- c(rep('B', N), rep('A', N))

str(factor(group)) #default levels mod <- multipleGroup(dat, 1, group = group, invariance = c(colnames(dat)[1:5], 'free_means', 'free_var'))

extract.mirt(mod, 'groupNames') coef(mod, simplify=TRUE) # factor A comes first in output

fgroup <- factor(group, levels = c('B', 'A')) str(fgroup) #default levels mod2 <- multipleGroup(dat, 1, group = fgroup, invariance = c(colnames(dat)[1:5], 'free_means', 'free_var'))

extract.mirt(mod2, 'groupNames') coef(mod2, simplify=TRUE) # factor A comes first in output

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/philchalmers/mirt/issues/193#issuecomment-689689723, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACA5DEK367MG2CICO56W2SLSE6XP3ANCNFSM4RCRUVRQ .

awmeade commented 4 years ago

OK, I made some changes for which I'll use a pull request.

  1. I switched the colors red and black. Before the reference group (group 1) was in red and the focal group (group 2) in black. Given that DIF is being evaluated with respect to the focal group, I thought it made more sense that the focal group be highlighted in red as generally the user will want to know how the focal group compares to the reference group. There are three places in the empirical_ES code where this appears in case you want to discard this change.
  2. Here is the code I wrote to grab the group labels from extract.mirt. I use unique and 'group' to get the group labels in the order in which they appear based on the character array assignment. Then, I'm validating for length. I did several tests switching around the labels to make sure that alphabetization was not an issue. I didn't make any default group labels, but I am assuming that's happening elsewhere.

grps <- unique(extract.mirt(mod, 'group')) if(nchar(as.character(grps[1])) > 30 || nchar(as.character(grps[2])) > 30){ grps <- c("ref","foc") } plot.df1 <- data.frame(Theta=Theta.focal[,1],ETS=ETS.foc.obs,group=as.character(grps[1])) plot.df2 <- data.frame(Theta=Theta.focal[,1],ETS=ETS.ref.obs,group=as.character(grps[2]))

If this looks OK to you, I'll use a pull request in github and you can refer and test further. You are a much better R coder than I am, so there may be a better way to do this. Feel free to discard all or whatever you would like with this!

adam

On Wed, Sep 9, 2020 at 3:02 PM Adam Meade awmeademail@gmail.com wrote:

scratch that.... still working

On Wed, Sep 9, 2020 at 2:56 PM Adam Meade awmeademail@gmail.com wrote:

I'm thinking maybe: grps <- table(extract.mirt(mod, 'group')) grps[1] would be best to get group labels

it appears that extract.mirt('groups') always pulls things in the order in which they appear which may be a lot simpler than pulling 'groupNames' which may or may not reflect the order of the datalabels.

Adam

On Wed, Sep 9, 2020 at 1:57 PM Adam Meade awmeademail@gmail.com wrote:

Interesting, and I see what you are saying.

I'll play around with this a bit more....

Adam

On Wed, Sep 9, 2020 at 12:55 PM Phil Chalmers notifications@github.com wrote:

Second, I've taken a stab at using extract.mirt(mod, 'groupnames') in the empirical_ES function and have realized that the extract.mirt function always returns the group names in alphabetical order.

That's not strictly true, it's actually based on how the factor() function works. If group is supplied as a character vector initial then it is passed to factor(), which dictates the group ordering. If you wanted group B, for instance, to be the control group and group A to be the focal, then you need to initially create group as a different ordered factor. See below for an example of this in case it helps in your code:

library(mirt)

no DIF

set.seed(12345)a <- matrix(abs(rnorm(15,1,.3)), ncol=1)d <- matrix(rnorm(15,0,.7),ncol=1)temtype <- rep('2PL', nrow(a))N <- 1000dataset1 <- simdata(a, d, N, itemtype)dataset2 <- simdata(a, d, N, itemtype, mu = .1, sigma = matrix(1.5))dat <- rbind(dataset1, dataset2)group <- c(rep('B', N), rep('A', N))

str(factor(group)) #default levels mod <- multipleGroup(dat, 1, group = group, invariance = c(colnames(dat)[1:5], 'free_means', 'free_var'))

extract.mirt(mod, 'groupNames') coef(mod, simplify=TRUE) # factor A comes first in output

fgroup <- factor(group, levels = c('B', 'A')) str(fgroup) #default levels mod2 <- multipleGroup(dat, 1, group = fgroup, invariance = c(colnames(dat)[1:5], 'free_means', 'free_var'))

extract.mirt(mod2, 'groupNames') coef(mod2, simplify=TRUE) # factor A comes first in output

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/philchalmers/mirt/issues/193#issuecomment-689689723, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACA5DEK367MG2CICO56W2SLSE6XP3ANCNFSM4RCRUVRQ .

philchalmers commented 4 years ago

Sure, fire away. I'll close this issue and we'll work with the pull request instead. Thanks!