HDF5 error

[AnjaliC4]

plotGroups(proj,colorBy="GeneIntegrationMatrix", name="PLP1", groupBy="Clusters", plotAs="violin",threads=1)

Error: Getting ImputeWeights No imputeWeights found, returning NULL 2022-10-14 19:54:41 : Getting Matrix Values... 1 2 3 4 5 6 7 8 9 10 <simpleError in H5Lexists(loc$H5Identifier, name): HDF5. Links. Can't get value.>

---

11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 Error in .safelapply(seq_along(cellNamesList), function(x) { : Error Found Iteration 10 : [1] "Error in .logError(e, fn = \".getMatFromArrow\", info = \"\", errorList = errorList, : \n Exiting See Error Above\n" <simpleError in .logError(e, fn = ".getMatFromArrow", info = "", errorList = errorList, logFile = logFile): Exiting See Error Above> In addition: Warning message: In mclapply(..., mc.cores = threads, mc.preschedule = preschedule) : 1 function calls resulted in an error

Apparently there is no issue in retreiving cellColData, but this issue persists when extracting any matrix from ArrowFiles.

[rcorces]

Hi @AnjaliC4! Thanks for using ArchR! Please make sure that your post belongs in the Issues section. Only bugs and error reports belong in the Issues section. Usage questions and feature requests should be posted in the Discussions section, not in Issues.
Before we help you, you must respond to the following questions unless your original post already contained this information: 1. If you've encountered an error, have you already searched previous Issues to make sure that this hasn't already been solved? 2. Can you recapitulate your error using the tutorial code and dataset? If so, provide a reproducible example. 3. Did you post your log file? If not, add it now. 4. Remove any screenshots that contain text and instead copy and paste the text using markdown's codeblock syntax (three consecutive backticks). You can do this by editing your original post.

[AnjaliC4]

Hi, the above issue started happening after I tried adding chromVar matrix which also throws an error.

proj <- addMotifAnnotations(proj,motifSet = "homer",name = "Homer") proj <- addBgdPeaks(proj, force=TRUE) proj <- addDeviationsMatrix(proj, peakAnnotation = "Homer",force=TRUE, threads =8)

Error: Error in h5ls(ArrowFiles[x]) : HDF5. Links. Iteration failed. Calls: addDeviationsMatrix ... .availableSeqnames -> .safelapply -> lapply -> FUN -> %>% -> h5ls Execution halted

[rcorces]

I'm not sure. This looks like your arrow files have been corrupted somehow. As requested in my initial reply, you did not provide your log file so I cant say any more.

[AnjaliC4]

Hi @rcorces, I used an uncorrupted version of my ArchR project and tried adding HOMER annotations as described in tutorial. Please see error and log file attached here. ArchR-addDeviationsMatrix-48c01d7b518-Date-2022-10-17_Time-03-23-08.log error.log

proj <- addMotifAnnotations(proj,motifSet = "homer",name = "Motif",force=TRUE) proj <- addBgdPeaks(proj,force=TRUE) proj <- addDeviationsMatrix(proj,peakAnnotation = "Motif",force=TRUE,threads =8))

[rcorces]

Theres nothing in the log or error that helps me figure out what might be going on. Maybe try downgrading back to the current master branch (release_1.0.2) and see if that works.

[AnjaliC4]

In the log file - this is the error: 2022-10-17 03:47:41 : ERROR Found in .computeDeviations for atac_A10B10 (6 of 42) LogFile = ArchRLogs/ArchR-addDeviationsMatrix-48c01d7b518-Date-2022-10-17_Time-03-23-08.log

<std::logic_error in deviations_cpp(X = Matrix::t(countsMatrix), B = backgroudPeaks, anno_mat = annotationsMatrix, expect = matrix(norm_expectation, ncol = 1), CpS = matrix(countsPerSample, ncol = 1), prefix = prefix, print_every = pe, verbose = verbose): SpMat::init(): requested size is too large; suggest to enable ARMA_64BIT_WORD>

[rcorces]

You are running into an issue specifically due to changes on the dev branch where we are attempting to implement more of the chromVar deviations infrastructure in c++. I've added a change to this code via 7d44bd5 on the dev_deviationsCPP branch that might fix your issue.

To install:

devtools::install_github("GreenleafLab/ArchR", ref="dev_deviationsCPP", repos = BiocManager::repositories())
#to unload a package and reload
detach("package:ArchR", unload=TRUE)
library(ArchR)

[rcorces]

@AnjaliC4 Please let me know if the above fix worked for you. If end users do not validate the changes that we make to the code base, then we just end up spending time addressing issues that never get fixed. This makes me less inclined to respond to future posts when the efforts put in are not met with equal effort from the end users.

[AnjaliC4]

Hi @rcorces , yes this worked. Thanks for your time.

[rcorces]

Perfect thanks for confirming! This has been merged into dev and will become part of the next release.

[rcorces]

@AnjaliC4 - For some reason, I'm not able to successfully install ArchR with the addition of the commit to chromvar.cpp

#define ARMA_64BIT_WORD 1

I'm not sure why but for now I've reverted this change on the dev branch via https://github.com/GreenleafLab/ArchR/pull/1723. If this install really succeed for you, please do let me know because I'm unclear why it is failing for me (with a c-based error of "undefined symbol"

[AnjaliC4]

Hi @rcorces, I looked into my scripts and it seems like ultimately the error was resolved through re-installation of master branch. Sorry, I got this mixed up with all the different things that I was trying to do to resolve the error. Thanks for re-checking on this.

[samhkim]

Hi @rcorces, a similar issue with ARMA_64BIT_WORD is still present when trying to add a DeviationMatrix on the dev branch.

[rcorces]

@samhkim - I'm happy to try to address this issue but I dont have access to a machine to test fixes. Could you try to install the dev_arma64 branch and let me know if that solves the problem?

devtools::install_github("GreenleafLab/ArchR", ref="dev_arma64", repos = BiocManager::repositories(), upgrade = "never")
#to unload a package and reload
detach("package:ArchR", unload=TRUE)
library(ArchR)

[samhkim]

@rcorces thank you for a quick reply. I've tested the dev_arma64 branch and it seems to not run into the same error. I will update if I encounter any other errors afterwards.

[rcorces]

@samhkim - great thanks for confirming. I've merged this code change into the dev branch. Closing for now but let me know if related issues crop up.