Software Compatibility Issue Between phyloseq and lme4 Packages

[FrederickHuangLin]

Hi,

I've encountered a rather peculiar software issue involving the phyloseq package and the lme4 package when analyzing longitudinal microbiome data. Below, please find a reproducible example that triggers the error:

First, when I run the mixed effects model code provided by lme4, it works perfectly:

data("sleepstudy", package = "lme4")
m <- lme4::lmer(Reaction ~ Days + (Days | Subject), sleepstudy)
m

Linear mixed model fit by REML ['lmerModLmerTest']
Formula: Reaction ~ Days + (Days | Subject)
   Data: sleepstudy
REML criterion at convergence: 1743.628
Random effects:
 Groups   Name        Std.Dev. Corr
 Subject  (Intercept) 24.741       
          Days         5.922   0.07
 Residual             25.592       
Number of obs: 180, groups:  Subject, 18
Fixed Effects:
(Intercept)         Days  
     251.41        10.47

The above code runs without issues. However, when I load the phyloseq package and then attempt to run the same lme4 code, it results in errors:

library(phyloseq)
m <- lme4::lmer(Reaction ~ Days + (Days | Subject), sleepstudy)

Error in t(do.call(sparseMatrix, do.call(rbind, lapply(seq_along(blist),  : 
  invalid 'lazy' to R_sparse_transpose()

Even detaching the phyloseq package doesn't resolve the issue and leads to a different error message:

detach("package:phyloseq", unload = TRUE)
m <- lme4::lmer(Reaction ~ Days + (Days | Subject), sleepstudy)

Error in t.default(do.call(sparseMatrix, do.call(rbind, lapply(seq_along(blist),  : 
                                                                   argument is not a matrix

As of now, the only solution I've found is to restart the R session. Below is my sessionInfo():

 version 4.3.1 (2023-06-16)
Platform: x86_64-apple-darwin20 (64-bit)
Running under: macOS Ventura 13.4.1

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
 [1] gtable_0.3.4            ggplot2_3.4.3           rhdf5_2.44.0            Biobase_2.60.0          lattice_0.21-8         
 [6] rhdf5filters_1.12.1     vctrs_0.6.3             tools_4.3.1             bitops_1.0-7            generics_0.1.3         
[11] biomformat_1.28.0       stats4_4.3.1            parallel_4.3.1          tibble_3.2.1            fansi_1.0.4            
[16] cluster_2.1.4           pkgconfig_2.0.3         Matrix_1.6-1            data.table_1.14.8       S4Vectors_0.38.1       
[21] lifecycle_1.0.3         GenomeInfoDbData_1.2.10 compiler_4.3.1          stringr_1.5.0           Biostrings_2.68.1      
[26] munsell_0.5.0           codetools_0.2-19        permute_0.9-7           GenomeInfoDb_1.36.3     RCurl_1.98-1.12        
[31] pillar_1.9.0            nloptr_2.0.3            crayon_1.5.2            MASS_7.3-60             vegan_2.6-4            
[36] iterators_1.0.14        boot_1.3-28.1           foreach_1.5.2           nlme_3.1-163            tidyselect_1.2.0       
[41] digest_0.6.33           stringi_1.7.12          dplyr_1.1.3             reshape2_1.4.4          splines_4.3.1          
[46] ade4_1.7-22             grid_4.3.1              colorspace_2.1-0        cli_3.6.1               magrittr_2.0.3         
[51] survival_3.5-7          utf8_1.2.3              ape_5.7-1               scales_1.2.1            XVector_0.40.0         
[56] igraph_1.5.1            multtest_2.56.0         lme4_1.1-34             IRanges_2.34.1          mgcv_1.9-0             
[61] rlang_1.1.1             Rcpp_1.0.11             glue_1.6.2              BiocGenerics_0.46.0     rstudioapi_0.15.0      
[66] minqa_1.2.6             jsonlite_1.8.7          R6_2.5.1                Rhdf5lib_1.22.1         plyr_1.8.8             
[71] zlibbioc_1.46.0 

Also, I using Bioconductor 3.17.

Any insights or advice on resolving this issue would be greatly appreciated.

Best regards,

Huang

[MelanieCHay]

I am encountering something similar when running NetCoMi and phyloseq.

"Error in t(beta) : invalid 'lazy' to R_sparse_transpose()"

[szymanskishay]

EDIT: Actually I just realized, if the issue is what I detail below, if you load lme4 before phyloseq that might fix it? you might have to mess with your .RData/workspace to get this to work if your workspace is automatically loading phyloseq on start though.

As you suggested, the solution I have found for this same compatibility issue that reliably works is really just restarting everything. I suspect, but cannot confirm entirely, that there is a daisy-chain of dependencies and loaded name spaces that makes it hard to entirely fix. I think phyloseq loads things that need a different version of Matrix and if it is already loaded, then Matrix is already loaded as a namespace, so attaching Matrix or reloading it from other sources doesn't set to the more recent version thats actually being used. What I have found useful for my personal situation is to save whatever tables are being used as files to the computer itself, then reload them in a different script after starting a new R session with a clear workspace. I also have had success in the past when I encountered this before by progressively unloading every conflicting dependency and then reattaching the package im trying to use that is calling Matrix (so in your case, lme4) so that it re-attaches the right one. This can be very frustrating to figure out. If I remember right, vegan also can do this, but I am not sure if its because I have been using vegan in the same sessions as phyloseq. I think BiocGenerics and things it calls are potentially problematic. I remember having to remove survival as well.

[SJohnsonMayo]

Encountering this same issue, except it is causing R to segfault. Loading other packages before phyloseq doesn't seem to fix it.

[minsiksudo]

Check out his thread. It worked for my setting.

https://support.bioconductor.org/p/9154277/