search

GreenleafLab / ArchR

ArchR : Analysis of Regulatory Chromatin in R (www.ArchRProject.com)
MIT License
382 stars 136 forks source link

addGeneIntegrationMatrix;addToArrow = TRUE #1035

Closed ZhengGongwei closed 3 years ago

ZhengGongwei commented 3 years ago

when i used addGeneIntegrationMatrix(），i got error：

Error in .safelapply(seq_along(allSamples), function(y) { : Error Found Iteration 2 : [1] "Error in H5Fopen(file) : HDF5. File accessibility. Unable to open file.\n" <simpleError in H5Fopen(file): HDF5. File accessibility. Unable to open file.> Error Found Iteration 3 : [1] "Error in H5Fopen(file) : HDF5. File accessibility. Unable to open file.\n" <simpleError in H5Fopen(file): HDF5. File accessibility. Unable to open file.> In addition: Warning messages: 1: In sprintf("Completed Getting Magic Weights!", round(object.size(weightList)/10^9, : one argument not used by format 'Completed Getting Magic Weights!' 2: In mclapply(..., mc.cores = threads, mc.preschedule = preschedule) : 2 function calls resulted in an error

projHeme3 <- addGeneIntegrationMatrix( ArchRProj = projHeme2, useMatrix = "GeneScoreMatrix", matrixName = "GeneIntegrationMatrix", reducedDims = "IterativeLSI", seRNA = seRNA, addToArrow = TRUE, force= TRUE, groupList = groupList, groupRNA = "BioClassification", nameCell = "predictedCell", nameGroup = "predictedGroup", nameScore = "predictedScore" )

ArchR logging to : ArchRLogs/ArchR-addGeneIntegrationMatrix-58f9be149a8-Date-2021-09-15_Time-09-28-51.log If there is an issue, please report to github with logFile! 2021-09-15 09:28:52 : Running Seurat's Integration Stuart* et al 2019, 0.002 mins elapsed. 2021-09-15 09:28:52 : Checking ATAC Input, 0.007 mins elapsed. 2021-09-15 09:28:52 : Checking RNA Input, 0.007 mins elapsed. 2021-09-15 09:28:57 : Found 18601 overlapping gene names from gene scores and rna matrix!, 0.091 mins elapsed. 2021-09-15 09:28:57 : Creating Integration Blocks, 0.091 mins elapsed. 2021-09-15 09:28:57 : Prepping Interation Data, 0.095 mins elapsed. 2021-09-15 09:28:58 : Computing Integration in 1 Integration Blocks!, 0 mins elapsed. 2021-09-15 09:28:58 : Block (1 of 1) : Computing Integration, 0 mins elapsed. 2021-09-15 09:28:59 : Block (1 of 1) : Identifying Variable Genes, 0.023 mins elapsed. 2021-09-15 09:29:01 : Block (1 of 1) : Getting GeneScoreMatrix, 0.058 mins elapsed. 2021-09-15 09:29:11 : Block (1 of 1) : Imputing GeneScoreMatrix, 0.228 mins elapsed. Getting ImputeWeights Warning: The following arguments are not used: row.names 2021-09-15 09:29:48 : Block (1 of 1) : Seurat FindTransferAnchors, 0.833 mins elapsed. 2021-09-15 09:31:19 : Block (1 of 1) : Seurat TransferData Cell Group Labels, 2.35 mins elapsed. 2021-09-15 09:31:21 : Block (1 of 1) : Seurat TransferData Cell Names Labels, 2.385 mins elapsed. 2021-09-15 09:31:39 : Block (1 of 1) : Seurat TransferData GeneMatrix, 2.688 mins elapsed. 2021-09-15 09:31:47 : Block (1 of 1) : Saving TransferAnchors Joint CCA, 2.823 mins elapsed. 2021-09-15 09:31:48 : Block (1 of 1) : Transferring Paired RNA to Temp File, 2.842 mins elapsed. 2021-09-15 09:32:06 : Block (1 of 1) : Completed Integration, 3.133 mins elapsed. 2021-09-15 09:32:06 : Block (1 of 1) : Plotting Joint UMAP, 3.145 mins elapsed. Length of unique values greater than palette, interpolating.. 2021-09-15 09:32:53 : Transferring Data to ArrowFiles, 3.922 mins elapsed. 2021-09-15 09:32:53 : scATAC_BMMC_R1 (1 of 3) Getting GeneIntegrationMatrix From TempFiles!, 3.923 mins elapsed. 2021-09-15 09:32:53 : scATAC_CD34_BMMC_R1 (2 of 3) Getting GeneIntegrationMatrix From TempFiles!, 3.924 mins elapsed. 2021-09-15 09:32:53 : scATAC_PBMC_R1 (3 of 3) Getting GeneIntegrationMatrix From TempFiles!, 3.925 mins elapsed. 2021-09-15 09:33:58 : scATAC_BMMC_R1 (1 of 3) Adding GeneIntegrationMatrix to ArrowFile!, 5.01 mins elapsed. Error in .safelapply(seq_along(allSamples), function(y) { : Error Found Iteration 2 : [1] "Error in H5Fopen(file) : HDF5. File accessibility. Unable to open file.\n" <simpleError in H5Fopen(file): HDF5. File accessibility. Unable to open file.> Error Found Iteration 3 : [1] "Error in H5Fopen(file) : HDF5. File accessibility. Unable to open file.\n" <simpleError in H5Fopen(file): HDF5. File accessibility. Unable to open file.> In addition: Warning messages: 1: In sprintf("Completed Getting Magic Weights!", round(object.size(weightList)/10^9, : one argument not used by format 'Completed Getting Magic Weights!' 2: In mclapply(..., mc.cores = threads, mc.preschedule = preschedule) : 2 function calls resulted in an error

       ___      .______        ______  __    __  .______      
      /   \     |   _  \      /      ||  |  |  | |   _  \     
     /  ^  \    |  |_)  |    |  ,----'|  |__|  | |  |_)  |    
    /  /_\  \   |      /     |  |     |   __   | |      /     
   /  _____  \  |  |\  \\___ |  `----.|  |  |  | |  |\  \\___.
  /__/     \__\ | _| `._____| \______||__|  |__| | _| `._____|

Logging With ArchR!

Start Time : 2021-09-15 09:28:51

------- ArchR Info

ArchRThreads = 16 ArchRGenome = Hg19

------- System Info

Computer OS = unix Total Cores = 40

------- Session Info

R version 4.1.0 (2021-05-18) Platform: x86_64-conda-linux-gnu (64-bit) Running under: CentOS Linux 7 (Core)

Matrix products: default BLAS/LAPACK: /share2/pub/zhenggw/zhenggw/anaconda3/envs/R410/lib/libopenblasp-r0.3.17.so

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] parallel stats4 stats graphics grDevices utils datasets [8] methods base

other attached packages: [1] gtable_0.3.0 SeuratObject_4.0.2
[3] Seurat_4.0.4 ArchR_1.0.1
[5] magrittr_2.0.1 rhdf5_2.37.0
[7] Matrix_1.3-4 data.table_1.14.0
[9] SummarizedExperiment_1.23.4 Biobase_2.53.0
[11] GenomicRanges_1.45.0 GenomeInfoDb_1.29.4
[13] IRanges_2.27.2 S4Vectors_0.31.3
[15] BiocGenerics_0.39.2 MatrixGenerics_1.5.4
[17] matrixStats_0.60.1 ggplot2_3.3.5

loaded via a namespace (and not attached): [1] plyr_1.8.6 igraph_1.2.6
[3] lazyeval_0.2.2 splines_4.1.0
[5] BiocParallel_1.27.4 listenv_0.8.0
[7] scattermore_0.7 digest_0.6.27
[9] htmltools_0.5.2 fansi_0.5.0
[11] BSgenome_1.61.0 tensor_1.5
[13] cluster_2.1.2 ROCR_1.0-11
[15] globals_0.14.0 Biostrings_2.61.2
[17] spatstat.sparse_2.0-0 colorspace_2.0-2
[19] ggrepel_0.9.1 dplyr_1.0.7
[21] crayon_1.4.1 RCurl_1.98-1.4
[23] jsonlite_1.7.2 spatstat.data_2.1-0
[25] survival_3.2-13 zoo_1.8-9
[27] glue_1.4.2 polyclip_1.10-0
[29] zlibbioc_1.39.0 XVector_0.33.0
[31] leiden_0.3.9 DelayedArray_0.19.2
[33] Rhdf5lib_1.15.2 future.apply_1.8.1
[35] abind_1.4-5 scales_1.1.1
[37] DBI_1.1.1 miniUI_0.1.1.1
[39] Rcpp_1.0.7 viridisLite_0.4.0
[41] xtable_1.8-4 reticulate_1.20
[43] spatstat.core_2.3-0 htmlwidgets_1.5.3
[45] httr_1.4.2 RColorBrewer_1.1-2
[47] nabor_0.5.0 ellipsis_0.3.2
[49] ica_1.0-2 farver_2.1.0
[51] pkgconfig_2.0.3 XML_3.99-0.7
[53] uwot_0.1.10 deldir_0.2-10
[55] utf8_1.2.2 labeling_0.4.2
[57] tidyselect_1.1.1 rlang_0.4.11
[59] reshape2_1.4.4 later_1.3.0
[61] munsell_0.5.0 tools_4.1.0
[63] generics_0.1.0 ggridges_0.5.3
[65] stringr_1.4.0 fastmap_1.1.0
[67] yaml_2.2.1 goftest_1.2-2
[69] fitdistrplus_1.1-5 purrr_0.3.4
[71] RANN_2.6.1 pbapply_1.4-3
[73] future_1.22.1 nlme_3.1-152
[75] mime_0.11 compiler_4.1.0
[77] rstudioapi_0.13 plotly_4.9.4.1
[79] png_0.1-7 spatstat.utils_2.2-0
[81] tibble_3.1.4 stringi_1.7.3
[83] RSpectra_0.16-0 lattice_0.20-44
[85] vctrs_0.3.8 pillar_1.6.2
[87] lifecycle_1.0.0 rhdf5filters_1.5.0
[89] spatstat.geom_2.2-2 lmtest_0.9-38
[91] RcppAnnoy_0.0.19 cowplot_1.1.1
[93] bitops_1.0-7 irlba_2.3.3
[95] httpuv_1.6.2 patchwork_1.1.1
[97] rtracklayer_1.53.1 R6_2.5.1
[99] BiocIO_1.3.0 promises_1.2.0.1
[101] KernSmooth_2.23-20 gridExtra_2.3
[103] parallelly_1.27.0 codetools_0.2-18
[105] gtools_3.9.2 MASS_7.3-54
[107] assertthat_0.2.1 rjson_0.2.20
[109] withr_2.4.2 GenomicAlignments_1.29.0
[111] sctransform_0.3.2 Rsamtools_2.9.1
[113] GenomeInfoDbData_1.2.6 mgcv_1.8-36
[115] grid_4.1.0 rpart_4.1-15
[117] tidyr_1.1.3 BSgenome.Hsapiens.UCSC.hg19_1.4.3 [119] Cairo_1.5-12.2 Rtsne_0.15
[121] shiny_1.6.0 restfulr_0.0.13

------- Log Info

2021-09-15 09:28:52 : Running Seurat's Integration Stuart* et al 2019, 0.002 mins elapsed.

2021-09-15 09:28:52 : Input-Parameters, Class = list

Input-Parameters$: length = 1

1 function (name)
2 .Internal(args(name))

Input-Parameters$ArchRProj: length = 1

Input-Parameters$useMatrix: length = 1 [1] "GeneScoreMatrix"

Input-Parameters$matrixName: length = 1 [1] "GeneIntegrationMatrix"

Input-Parameters$reducedDims: length = 1 [1] "IterativeLSI"

Input-Parameters$seRNA: length = 20287 class: RangedSummarizedExperiment dim: 6 35582 metadata(0): assays(1): counts rownames(6): FAM138A OR4F5 ... OR4F16 FAM87B rowData names(3): gene_name gene_id exonLength colnames(35582): CD34_32_R5:AAACCTGAGTATCGAA-1 CD34_32_R5:AAACCTGAGTCGTTTG-1 ... BMMC_10x_GREENLEAF_REP2:TTTGTTGCATGTGTCA-1 BMMC_10x_GREENLEAF_REP2:TTTGTTGCATTGAAAG-1 colData names(10): Group nUMI_pre ... BioClassification Barcode

Input-Parameters$groupATAC: length = 0 NULL

Input-Parameters$groupRNA: length = 1 [1] "BioClassification"

Input-Parameters$groupList: length = 0 NULL

Input-Parameters$sampleCellsATAC: length = 1 [1] 10000

Input-Parameters$sampleCellsRNA: length = 1 [1] 10000

Input-Parameters$embeddingATAC: length = 0 NULL

Input-Parameters$embeddingRNA: length = 0 NULL

Input-Parameters$dimsToUse: length = 30 [1] 1 2 3 4 5 6

Input-Parameters$scaleDims: length = 0 NULL

Input-Parameters$corCutOff: length = 1 [1] 0.75

Input-Parameters$plotUMAP: length = 1 [1] TRUE

Input-Parameters$nGenes: length = 1 [1] 2000

Input-Parameters$useImputation: length = 1 [1] TRUE

Input-Parameters$reduction: length = 1 [1] "cca"

Input-Parameters$addToArrow: length = 1 [1] TRUE

Input-Parameters$scaleTo: length = 1 [1] 10000

Input-Parameters$genesUse: length = 0 NULL

Input-Parameters$nameCell: length = 1 [1] "predictedCell_Un"

Input-Parameters$nameGroup: length = 1 [1] "predictedGroup_Un"

Input-Parameters$nameScore: length = 1 [1] "predictedScore_Un"

Input-Parameters$threads: length = 1 [1] 16

Input-Parameters$verbose: length = 1 [1] TRUE

Input-Parameters$force: length = 1 [1] TRUE

Input-Parameters$logFile: length = 1 [1] "ArchRLogs/ArchR-addGeneIntegrationMatrix-58f9be149a8-Date-2021-09-15_Time-09-28-51.log"

2021-09-15 09:28:52 : Checking ATAC Input, 0.007 mins elapsed. 2021-09-15 09:28:52 : Checking RNA Input, 0.007 mins elapsed. 2021-09-15 09:28:57 : Found 18601 overlapping gene names from gene scores and rna matrix!, 0.091 mins elapsed. 2021-09-15 09:28:57 : Creating Integration Blocks, 0.091 mins elapsed. 2021-09-15 09:28:57 : Prepping Interation Data, 0.095 mins elapsed. 2021-09-15 09:28:58 : Computing Integration in 1 Integration Blocks!, 0 mins elapsed. 2021-09-15 09:28:58 : Block (1 of 1) : Computing Integration, 0 mins elapsed. 2021-09-15 09:28:59 : Block (1 of 1) : Identifying Variable Genes, 0.023 mins elapsed. 2021-09-15 09:29:01 : Block (1 of 1) : Getting GeneScoreMatrix, 0.058 mins elapsed.

2021-09-15 09:29:11 : GeneScoreMat-Block-1, Class = dgCMatrix GeneScoreMat-Block-1: nRows = 2000, nCols = 10250 GeneScoreMat-Block-1: NonZeroEntries = 3681420, EntryRange = [ 0.022 , 229.174 ] 5 x 5 sparse Matrix of class "dgCMatrix" scATAC_BMMC_R1#TCACCTGTCCTCCTGA-1 scATAC_PBMC_R1#TCTCTGGGTCATCAAC-1 HES4 . .
ISG15 . .
TNFRSF18 4.124 .
TNFRSF4 . .
CALML6 . 1.751 scATAC_CD34_BMMC_R1#TGCCTCATCTCGACCT-1 HES4 2.188 ISG15 .
TNFRSF18 .
TNFRSF4 .
CALML6 .
scATAC_PBMC_R1#TGATTTCTCATTGCCC-1 scATAC_PBMC_R1#GCTCAGGTCGCGTGAC-1 HES4 . . ISG15 . . TNFRSF18 . . TNFRSF4 7.228 . CALML6 . .

2021-09-15 09:29:11 : Block (1 of 1) : Imputing GeneScoreMatrix, 0.228 mins elapsed.

2021-09-15 09:29:11 : addImputeWeights Input-Parameters, Class = list

addImputeWeights Input-Parameters$ArchRProj: length = 1

addImputeWeights Input-Parameters$reducedDims: length = 1 [1] "IterativeLSI"

addImputeWeights Input-Parameters$dimsToUse: length = 30 [1] 1 2 3 4 5 6

addImputeWeights Input-Parameters$scaleDims: length = 0 NULL

addImputeWeights Input-Parameters$corCutOff: length = 1 [1] 0.75

addImputeWeights Input-Parameters$td: length = 1 [1] 3

addImputeWeights Input-Parameters$ka: length = 1 [1] 4

addImputeWeights Input-Parameters$sampleCells: length = 1 [1] 5000

addImputeWeights Input-Parameters$nRep: length = 1 [1] 2

addImputeWeights Input-Parameters$k: length = 1 [1] 15

addImputeWeights Input-Parameters$epsilon: length = 1 [1] 1

addImputeWeights Input-Parameters$useHdf5: length = 1 [1] TRUE

addImputeWeights Input-Parameters$randomSuffix: length = 1 [1] TRUE

addImputeWeights Input-Parameters$threads: length = 1 [1] 1

addImputeWeights Input-Parameters$seed: length = 1 [1] 1

addImputeWeights Input-Parameters$verbose: length = 1 [1] TRUE

addImputeWeights Input-Parameters$logFile: length = 1 [1] "ArchRLogs/ArchR-addGeneIntegrationMatrix-58f9be149a8-Date-2021-09-15_Time-09-28-51.log"

2021-09-15 09:29:11 : Computing Impute Weights Using Magic (Cell 2018), 0 mins elapsed. 2021-09-15 09:29:11 : Computing Partial Diffusion Matrix with Magic (1 of 2), 0.001 mins elapsed. 2021-09-15 09:29:19 : Computing Partial Diffusion Matrix with Magic (2 of 2), 0.124 mins elapsed. 2021-09-15 09:29:27 : Completed Getting Magic Weights!, 0.263 mins elapsed.

2021-09-15 09:29:27 : imputeMatrix Input-Parameters, Class = list

imputeMatrix Input-Parameters$mat: nRows = 2000, nCols = 10250 imputeMatrix Input-Parameters$mat: NonZeroEntries = 3681420, EntryRange = [ 0.022 , 229.174 ] 5 x 5 sparse Matrix of class "dgCMatrix" scATAC_BMMC_R1#TCACCTGTCCTCCTGA-1 scATAC_PBMC_R1#TCTCTGGGTCATCAAC-1 HES4 . .
ISG15 . .
TNFRSF18 4.124 .
TNFRSF4 . .
CALML6 . 1.751 scATAC_CD34_BMMC_R1#TGCCTCATCTCGACCT-1 HES4 2.188 ISG15 .
TNFRSF18 .
TNFRSF4 .
CALML6 .
scATAC_PBMC_R1#TGATTTCTCATTGCCC-1 scATAC_PBMC_R1#GCTCAGGTCGCGTGAC-1 HES4 . . ISG15 . . TNFRSF18 . . TNFRSF4 7.228 . CALML6 . .

imputeMatrix Input-Parameters$threads: length = 1 [1] 16

imputeMatrix Input-Parameters$verbose: length = 1 [1] FALSE

imputeMatrix Input-Parameters$logFile: length = 1 [1] "ArchRLogs/ArchR-addGeneIntegrationMatrix-58f9be149a8-Date-2021-09-15_Time-09-28-51.log"

2021-09-15 09:29:27 : mat, Class = dgCMatrix mat: nRows = 2000, nCols = 10250 mat: NonZeroEntries = 3681420, EntryRange = [ 0.022 , 229.174 ] 5 x 5 sparse Matrix of class "dgCMatrix" scATAC_BMMC_R1#TCACCTGTCCTCCTGA-1 scATAC_PBMC_R1#TCTCTGGGTCATCAAC-1 HES4 . .
ISG15 . .
TNFRSF18 4.124 .
TNFRSF4 . .
CALML6 . 1.751 scATAC_CD34_BMMC_R1#TGCCTCATCTCGACCT-1 HES4 2.188 ISG15 .
TNFRSF18 .
TNFRSF4 .
CALML6 .
scATAC_PBMC_R1#TGATTTCTCATTGCCC-1 scATAC_PBMC_R1#GCTCAGGTCGCGTGAC-1 HES4 . . ISG15 . . TNFRSF18 . . TNFRSF4 7.228 . CALML6 . .

2021-09-15 09:29:27 : weightList, Class = SimpleList

weightList$w1: length = 1 [1] "HemeTutorial/ImputeWeights/Impute-Weights-58f9b77a4488d-Date-2021-09-15_Time-09-29-11-Rep-1"

weightList$w2: length = 1 [1] "HemeTutorial/ImputeWeights/Impute-Weights-58f9b77a4488d-Date-2021-09-15_Time-09-29-11-Rep-2"

2021-09-15 09:29:27 : Imputing Matrix (1 of 2), 0 mins elapsed.

2021-09-15 09:29:27 :

2021-09-15 09:29:27 :

2021-09-15 09:29:27 : 2021-09-15 09:29:34 : Imputing Matrix (2 of 2), 0.105 mins elapsed.

2021-09-15 09:29:34 :

2021-09-15 09:29:34 :

2021-09-15 09:29:34 : 2021-09-15 09:29:42 : Finished Imputing Matrix, 0.251 mins elapsed.

2021-09-15 09:29:42 : GeneScoreMat-Block-Impute-1, Class = dgeMatrix GeneScoreMat-Block-Impute-1: nRows = 2000, nCols = 10250 GeneScoreMat-Block-Impute-1: NonZeroEntries = 20500000, EntryRange = [ 0 , 9.15439843748967 ] 5 x 5 Matrix of class "dgeMatrix" scATAC_BMMC_R1#TCACCTGTCCTCCTGA-1 scATAC_PBMC_R1#TCTCTGGGTCATCAAC-1 HES4 0.5184427 1.0362254 ISG15 0.6171816 0.7702490 TNFRSF18 0.8151052 1.2211798 TNFRSF4 0.3067044 0.7579912 CALML6 1.7554042 1.2784952 scATAC_CD34_BMMC_R1#TGCCTCATCTCGACCT-1 HES4 1.1514755 ISG15 0.4619424 TNFRSF18 0.8485447 TNFRSF4 0.5834297 CALML6 0.9720259 scATAC_PBMC_R1#TGATTTCTCATTGCCC-1 scATAC_PBMC_R1#GCTCAGGTCGCGTGAC-1 HES4 0.7774322 0.9136392 ISG15 0.6565110 0.7030960 TNFRSF18 0.6695197 1.0228874 TNFRSF4 0.4762739 0.5835516 CALML6 1.0334223 1.3246475

2021-09-15 09:29:48 : Block (1 of 1) : Seurat FindTransferAnchors, 0.833 mins elapsed.

2021-09-15 09:31:19 : transferAnchors-1, Class = character

transferAnchors-1: length = 1 [1] "An AnchorSet object containing 1457 anchors between the reference and query Seurat objects. \n This can be used as input to TransferData."

2021-09-15 09:31:19 : rDSub-1, Class = matrix

2021-09-15 09:31:19 : rDSub-1, Class = array LSI1 LSI2 LSI3 scATAC_BMMC_R1#TCACCTGTCCTCCTGA-1 -3.930892 0.8467445 -0.6136641 scATAC_PBMC_R1#TCTCTGGGTCATCAAC-1 -4.907704 1.8237665 0.7398185 scATAC_CD34_BMMC_R1#TGCCTCATCTCGACCT-1 -4.334570 -0.5432949 -1.8547806 scATAC_PBMC_R1#TGATTTCTCATTGCCC-1 -4.813749 1.8277396 0.6429701 scATAC_PBMC_R1#GCTCAGGTCGCGTGAC-1 -4.747173 1.7879898 0.8362609 LSI4 LSI5 scATAC_BMMC_R1#TCACCTGTCCTCCTGA-1 -2.4496945 1.2914298 scATAC_PBMC_R1#TCTCTGGGTCATCAAC-1 0.3042194 -0.2275518 scATAC_CD34_BMMC_R1#TGCCTCATCTCGACCT-1 0.8161193 -0.1772381 scATAC_PBMC_R1#TGATTTCTCATTGCCC-1 0.2787490 0.2171253 scATAC_PBMC_R1#GCTCAGGTCGCGTGAC-1 0.2902240 0.1845279

rDSub-1: nRows = 10250, nCols = 30

2021-09-15 09:31:19 : Block (1 of 1) : Seurat TransferData Cell Group Labels, 2.35 mins elapsed. 2021-09-15 09:31:21 : Block (1 of 1) : Seurat TransferData Cell Names Labels, 2.385 mins elapsed. 2021-09-15 09:31:39 : Block (1 of 1) : Seurat TransferData GeneMatrix, 2.688 mins elapsed. 2021-09-15 09:31:47 : Block (1 of 1) : Saving TransferAnchors Joint CCA, 2.823 mins elapsed. 2021-09-15 09:31:48 : Block (1 of 1) : Transferring Paired RNA to Temp File, 2.842 mins elapsed. 2021-09-15 09:32:06 : Block (1 of 1) : Completed Integration, 3.133 mins elapsed. 2021-09-15 09:32:06 : Block (1 of 1) : Plotting Joint UMAP, 3.145 mins elapsed. 2021-09-15 09:32:53 : Transferring Data to ArrowFiles, 3.922 mins elapsed. 2021-09-15 09:32:53 : scATAC_BMMC_R1 (1 of 3) Getting GeneIntegrationMatrix From TempFiles!, 3.923 mins elapsed. 2021-09-15 09:32:53 : scATAC_CD34_BMMC_R1 (2 of 3) Getting GeneIntegrationMatrix From TempFiles!, 3.924 mins elapsed. 2021-09-15 09:32:53 : scATAC_PBMC_R1 (3 of 3) Getting GeneIntegrationMatrix From TempFiles!, 3.925 mins elapsed. 2021-09-15 09:33:58 : scATAC_BMMC_R1 (1 of 3) Adding GeneIntegrationMatrix to ArrowFile!, 5.01 mins elapsed. 2021-09-15 09:33:58 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (1 of 23)!, 5.01 mins elapsed. 2021-09-15 09:34:00 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (2 of 23)!, 5.039 mins elapsed. 2021-09-15 09:34:01 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (3 of 23)!, 5.06 mins elapsed. 2021-09-15 09:34:03 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (4 of 23)!, 5.095 mins elapsed. 2021-09-15 09:34:05 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (5 of 23)!, 5.115 mins elapsed. 2021-09-15 09:34:06 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (6 of 23)!, 5.138 mins elapsed. 2021-09-15 09:34:08 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (7 of 23)!, 5.168 mins elapsed. 2021-09-15 09:34:09 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (8 of 23)!, 5.185 mins elapsed. 2021-09-15 09:34:11 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (9 of 23)!, 5.221 mins elapsed. 2021-09-15 09:34:13 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (10 of 23)!, 5.258 mins elapsed. 2021-09-15 09:34:14 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (11 of 23)!, 5.277 mins elapsed. 2021-09-15 09:34:16 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (12 of 23)!, 5.302 mins elapsed. 2021-09-15 09:34:18 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (13 of 23)!, 5.337 mins elapsed. 2021-09-15 09:34:19 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (14 of 23)!, 5.355 mins elapsed. 2021-09-15 09:34:20 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (15 of 23)!, 5.373 mins elapsed. 2021-09-15 09:34:22 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (16 of 23)!, 5.402 mins elapsed. 2021-09-15 09:34:23 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (17 of 23)!, 5.428 mins elapsed. 2021-09-15 09:34:25 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (18 of 23)!, 5.45 mins elapsed. 2021-09-15 09:34:27 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (19 of 23)!, 5.484 mins elapsed. 2021-09-15 09:34:29 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (20 of 23)!, 5.515 mins elapsed. 2021-09-15 09:34:30 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (21 of 23)!, 5.537 mins elapsed. 2021-09-15 09:34:32 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (22 of 23)!, 5.571 mins elapsed. 2021-09-15 09:34:33 : Adding GeneIntegrationMatrix to scATAC_BMMC_R1 for Chr (23 of 23)!, 5.59 mins elapsed.

Thanks for your help!

rcorces commented 3 years ago

Looks like a duplicate of two other currently open issues. Closing. Feel free to comment here if you dont agree.

https://github.com/GreenleafLab/ArchR/issues/248 https://github.com/GreenleafLab/ArchR/issues/703

brianpenghe commented 2 years ago

I see a similar error. It may not be the same case as #248 and #703 as they use different functions? When I set addToArrow = FALSE the error is gone though.

Evi-050 commented 1 year ago

If it is still relevant: change the groupRNA = "orig.ident" and it works.

projHeme <- addGeneIntegrationMatrix( ArchRProj = projHeme, useMatrix = "GeneScoreMatrix", matrixName = "GeneIntegrationMatrix", reducedDims = "IterativeLSI", seRNA = seRNA, addToArrow = FALSE, groupRNA = "orig.ident", nameCell = "predictedCell_Un", nameGroup = "predictedGroup_Un", nameScore = "predictedScore_Un" )

dgagler commented 9 months ago

I'm running into a related issue while running addGeneIntegrationMatrix() with addToArrow = TRUE on some large datasets (407k cells in ArchR object and a Seurat object reference with 284k cells), except I'm not hitting any h5f errors but Rstudio is getting fatal errors and crashing when it begins transferring data to the arrow files. It gets through successfully when addToArrow = FALSE. Presumably this is due to some memory issue with the temp files. I've tried downsampling my Seurat object to 125k cells, setting threads = 1, and setting a large max global size in Rstudio but still no luck.

Any ideas for getting this to slide through? Or any advice on using ArchR on HPC environments?

archR <- addGeneIntegrationMatrix(
  ArchRProj = archR,
  useMatrix = "GeneScoreMatrix",
  matrixName = "GeneIntegrationMatrix",
  reducedDims = "Harmony_Batch",
  seRNA = seurat.down,
  addToArrow = FALSE,
  sampleCellsATAC = 7500,
  sampleCellsRNA = 7500,
  threads = 1,
  groupRNA = "majority_voting",
  nameCell = "predictedCell_Un",
  nameGroup = "predictedGroup_Un",
  nameScore = "predictedScore_Un",
  verbose = TRUE
)

Log file attached: ArchR-addGeneIntegrationMatrix-309a7ba7d4af-Date-2024-01-11_Time-01-44-22.log