Problem with singular covariance matrix

[karolinakamuda]

There is a problem with covariance matrix when regressors values' levels differ from one another. Problem appears in few recent versions of plm package.

Code below works in some older package versions (there is no error when running it with plm package version 2.2-2) but does not work in recent plm versions, including version 2.6-0.

example.data <- read.csv2("regressors.csv")

example.data$Date <- as.Date(example.data$Date)
example.data.plm <- plm::pdata.frame(example.data, index = c("Panel.units", "Date"))

formula <-  as.formula(paste("var1 ~", paste(colnames(example.data)[-c(1:3)], collapse = "+")))

model.obj <- plm::plm(formula, example.data.plm, model = "within")

model.summary.lst <- summary(model.obj)

regressors.csv

[tappek]

The error happens in pwaldtest (called by summary):

summary(model.obj) # errors
pwaldtest(model.obj) # errors
# Error in solve.default(vcov(x)[names.coefs_wo_int, names.coefs_wo_int],  : 
#                         system is computationally singular: reciprocal condition number = 5.70506e-18

This is due to the specific data constellation as the variance-covariance matrix is not invertible, more precisely this (simplified) code in pwaldtest yields the error: solve(vcov(model.obj)). When manipulating the tolerance, the matrix is invertible: solve(vcov(model.obj), tol = .Machine$double.eps^2)

Versions up to plm 2.2-2 used a different approach to calculate the Wald statistic ((tss-ssr)/(ssr/df2)) but we saw issues with that as well. I do not think we can do much about it. Manipulating the tolerance will likely yield to computational precission issues for other data constellations. Maybe some scaling of your variables or a slightly different model won't have the computational issues as the data constellation will be a different one.

[tappek]

Making as question and closing this - feel free to post your thoughts and re-open