satijalab / seurat

R toolkit for single cell genomics
http://www.satijalab.org/seurat
Other
2.27k stars 910 forks source link

IntegrateData crash when integrating large data set #7458

Closed joe-jhou2 closed 1 year ago

joe-jhou2 commented 1 year ago

Integrating 87K cells across 35 samples, using Norm+Scale method, not SCT. SelectIntegrationFeatures and FindIntegrationAnchors functions ran without problems. but IntegrateData was terminated by the error below: Error in .T2C(newTMat(i = c(ij1[, 1], ij2[, 1]), j = c(ij1[, 2], ij2[, : unable to coerce from TsparseMatrix to [CR]sparseMatrixwhen length of 'i' slot exceeds 2^31-1

Try on the cluster, requested 200G RAM and 16 cores. The terminal and R studio both return the same error as below.

R version 4.3.0 (2023-04-21) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 18.04.6 LTS

Matrix products: default BLAS/LAPACK: FlexiBLAS OPENBLAS; LAPACK version 3.10.1

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

time zone: America/Los_Angeles tzcode source: system (glibc)

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] SeuratObject_4.1.3 Seurat_4.3.0

loaded via a namespace (and not attached): [1] deldir_1.0-9 pbapply_1.7-0 gridExtra_2.3
[4] remotes_2.4.2 rlang_1.1.1 magrittr_2.0.3
[7] RcppAnnoy_0.0.20 matrixStats_1.0.0 ggridges_0.5.4
[10] compiler_4.3.0 spatstat.geom_3.2-1 callr_3.7.3
[13] png_0.1-8 vctrs_0.6.2 reshape2_1.4.4
[16] stringr_1.5.0 crayon_1.5.2 pkgconfig_2.0.3
[19] fastmap_1.1.1 ellipsis_0.3.2 utf8_1.2.3
[22] promises_1.2.0.1 ps_1.7.5 purrr_1.0.1
[25] jsonlite_1.8.5 goftest_1.2-3 later_1.3.1
[28] spatstat.utils_3.0-3 prettyunits_1.1.1 irlba_2.3.5.1
[31] parallel_4.3.0 cluster_2.1.4 R6_2.5.1
[34] ica_1.0-3 stringi_1.7.12 RColorBrewer_1.1-3
[37] spatstat.data_3.0-1 reticulate_1.30 parallelly_1.36.0
[40] lmtest_0.9-40 scattermore_1.2 Rcpp_1.0.10
[43] tensor_1.5 future.apply_1.11.0 zoo_1.8-12
[46] sctransform_0.3.5 httpuv_1.6.11 Matrix_1.5-4.1
[49] splines_4.3.0 igraph_1.4.3 tidyselect_1.2.0
[52] abind_1.4-5 codetools_0.2-19 spatstat.random_3.1-5 [55] miniUI_0.1.1.1 spatstat.explore_3.2-1 processx_3.8.1
[58] curl_5.0.1 pkgbuild_1.4.0 listenv_0.9.0
[61] lattice_0.21-8 tibble_3.2.1 plyr_1.8.8
[64] withr_2.5.0 shiny_1.7.4 ROCR_1.0-11
[67] Rtsne_0.16 desc_1.4.2 future_1.32.0
[70] survival_3.5-5 polyclip_1.10-4 fitdistrplus_1.1-11
[73] pillar_1.9.0 KernSmooth_2.23-21 plotly_4.10.2
[76] generics_0.1.3 rprojroot_2.0.3 sp_1.6-1
[79] ggplot2_3.4.2 munsell_0.5.0 scales_1.2.1
[82] globals_0.16.2 xtable_1.8-4 glue_1.6.2
[85] lazyeval_0.2.2 tools_4.3.0 data.table_1.14.8
[88] RANN_2.6.1 leiden_0.4.3 cowplot_1.1.1
[91] grid_4.3.0 tidyr_1.3.0 colorspace_2.1-0
[94] nlme_3.1-162 patchwork_1.1.2 cli_3.6.1
[97] spatstat.sparse_3.0-1 fansi_1.0.4 viridisLite_0.4.2
[100] dplyr_1.1.2 uwot_0.1.14 gtable_0.3.3
[103] digest_0.6.31 progressr_0.13.0 ggrepel_0.9.3
[106] htmlwidgets_1.6.2 htmltools_0.5.5 lifecycle_1.0.3
[109] httr_1.4.6 mime_0.12 MASS_7.3-60

# insert reproducible example here
joe-jhou2 commented 1 year ago

I saw a post mentioned Seurat v5 enable to handle large dataset integration. Unfortunately, install v5 consistently occurs this error:> remotes::install_github("satijalab/seurat", "seurat5", quiet = TRUE) These packages have more recent versions available. It is recommended to update all of them. Which would you like to update?

1: All
2: CRAN packages only
3: None
4: SeuratObject (4.1.3 -> 185884ae5...) [GitHub] 5: vctrs (0.6.2 -> 0.6.3 ) [CRAN]

Enter one or more numbers, or an empty line to skip updates: 3 Error: Failed to install 'Seurat' from GitHub: ! System command 'R' failed

saketkc commented 1 year ago

Hi @mimisikai, can you try with option 4(letting it update SeuratObject)? Please reopen if that does not fix it.