major processing time issue with lme()

[kendalynnm]

I would like to follow a stepwise model development approach, as demonstrated very clearly in Bliese & Ployhart 2002. However, adding a random effect which allows for slope variation among collars over time results in prohibitively long run time (allowed to run for 18 hrs but did not complete within that window).

Bliese&PloyhartOrganizationalResearchMethods.pdf

[bpbond]

What branch should I check out for this? Is it obvious where in the code you're fitting the model? Thanks

[kendalynnm]

The branch is FieldDataLME and the relevant file is LicorDataAnalysis, which is very streamline. Thank you!

[bpbond]

Hi @kendalynnm it looks to me that the problem occurs because you're trying to fit a nonsensical slope. Specifically,

CH4.2lme <- lme(nFCH4 ~ campaign,
             random = ~1|Collar,
             data = f_dat,
             na.action = na.omit)
#allows the slopes associated with campaign to vary randomly among Collar IDs
CH4.3 <- update(CH4.2lme, random = ~1+campaign|Collar)

says that the CH4.3 model can have slopes and intercepts that vary by Collar; but the fixed-effects part of the model is nFCH4 ~ campaign, and campaign is a factor, not numeric! I don't know why this causes lme to hang rather than report a problem, but fitting the same model using lmer makes the issue obvious:

CH4.2lmer <- lmer(nFCH4 ~ campaign + (1|Collar) + (campaign|Collar),
             data = f_dat,
             na.action=na.omit)

Error: number of observations (=610) <= number of random effects (=715) for 
term (campaign | Collar); the random-effects parameters and the residual 
variance (or scale parameter) are probably unidentifiable

If I understand correctly what you're trying to do, I think you need campaign to be numeric?

[bpbond]

[bpbond]

Two other notes:

Processing time: this is not a big dataset. Forget 18 hours; if you see a model-fitting taking 18 seconds there's a huge problem.

Also, the code has this note in it:

#anova() does not work with products of lme4

😕 this isn't true — you definitely can.