runSnapAddPmat Error

[RZiffra]

Hi Rongxin,

As discussed in the runMACS issue, I was able to call the cluster peaks successfully by updating to the most recent version of snaptools. However, I am now running into an issue on the next step when I try to add the peak x cell matrix to the snap object. I made sure I was running the latest version of Snaptools and SnapATAC before running. Here is the error:

runSnapAddPmat( obj=combined, tmp.folder=getwd(), peak=peak.gr, path.to.snaptools="/home/rziffra/.local/bin/snaptools", buffer.size=500 ) Epoch: checking input parameters ... Epoch: checking input parameters ... Epoch: adding cell-by-peak matrix into snap file ... Traceback (most recent call last): File "/home/rziffra/.local/bin/snaptools", line 38, in parse_args() File "/home/rziffra/.local/lib/python3.5/site-packages/snaptools/parser.py", line 176, in parse_args verbose=args.verbose) File "/home/rziffra/.local/lib/python3.5/site-packages/snaptools/add_pmat.py", line 145, in snap_pmat dump_read(snap_file, fout_frag.name, buffer_size, None, tmp_folder, True); File "/home/rziffra/.local/lib/python3.5/site-packages/snaptools/snap.py", line 836, in dump_read fout.write(("\t".join(map(str, item)) + "\n").encode()) TypeError: write() argument must be str, not bytes Error in runSnapAddPmatSingle(file, peak = peak, path.to.snaptools = path.to.snaptools, : 'runSnapAddPmat' call failed

[r3fang]

Hi Ryan, i thought you are using snaptools on python2.7

[r3fang]

did you have trouble for runMACS?

[RZiffra]

Hi Rongxin,

I didn't have any issues with runMACS.

I guess when I updated snaptools I actually installed in python3 instead of updating python2. I just removed all versions and installed the updated version in python2. I tried running runSnapAddPmat again - it was running for ~36hrs then threw this error:

runSnapAddPmat(

• obj=combined,
• tmp.folder=getwd(),
• peak=peak.gr,
• path.to.snaptools="/home/rziffra/.local/bin/snaptools",
• buffer.size=500
• ) Epoch: checking input parameters ... Epoch: checking input parameters ... Epoch: adding cell-by-peak matrix into snap file ... Epoch: checking input parameters ... Epoch: adding cell-by-peak matrix into snap file ... error: cell x peak matrix already exists, delete it using snap-del first Error in runSnapAddPmatSingle(file, peak = peak, path.to.snaptools = path.to.snaptools, : 'runSnapAddPmat' call failed

I think one of the failed runs before probably added a partial Peak x cell matrix that i need to clear first. I will try to delete these using snap-del and try running again. Will update shortly.

Ryan

[r3fang]

Ryan,

You are absolutely right about every step. It’s because one of the sample already has PM session. If the snap object is a combination of multiple sample, you can set nun.cores to more than sample number run this in parallel.

Sent from my iPhone

On Apr 4, 2019, at 5:29 PM, RZiffra notifications@github.com wrote:

Hi Rongxin,

I didn't have any issues with runMACS.

I guess when I updated snaptools I actually installed in python3 instead of updating python2. I just removed all versions and installed the updated version in python2. I tried running runSnapAddPmat again - it was running for ~36hrs then threw this error:

runSnapAddPmat(

obj=combined, tmp.folder=getwd(), peak=peak.gr, path.to.snaptools="/home/rziffra/.local/bin/snaptools", buffer.size=500 ) Epoch: checking input parameters ... Epoch: checking input parameters ... Epoch: adding cell-by-peak matrix into snap file ... Epoch: checking input parameters ... Epoch: adding cell-by-peak matrix into snap file ... error: cell x peak matrix already exists, delete it using snap-del first Error in runSnapAddPmatSingle(file, peak = peak, path.to.snaptools = path.to.snaptools, : 'runSnapAddPmat' call failed I think one of the failed runs before probably added a partial Peak x cell matrix that i need to clear first. I will try to delete these using snap-del and try running again. Will update shortly.

Ryan

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[RZiffra]

Hi Rongxin,

so runSnapAddPmat seems to have successfully completed after running snap-del to remove the old PM sessions. However, now when I run addPmatToSnap I get the following error:

combined = addPmatToSnap(obj=combined,num.cores=1)

caught segfault address 0x7ffd98d34b08, cause 'memory not mapped'

Traceback: 1: H5Dread(h5dataset = h5dataset, h5spaceFile = h5spaceFile, h5spaceMem = h5spaceMemompoundAsDataFrame = compoundAsDataFrame, drop = drop, ...) 2: doTryCatch(return(expr), name, parentenv, handler) 3: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 4: tryCatchList(expr, classes, parentenv, handlers) 5: tryCatch(expr, error = function(e) { call <- conditionCall(e) if (!is.null({ if (identical(call[[1L]], quote(doTryCatch))) call <- sys.call(- dcall <- deparse(call)[1L] prefix <- paste("Error in", dcall, ": ") - 75L sm <- strsplit(conditionMessage(e), "\n")[[1L]] w <- 14L + nchar(type = "w") + nchar(sm[1L], type = "w") if (is.na(w)) w <- 14L + nall, type = "b") + nchar(sm[1L], type = "b") if (w > LONG) prefix <- paste0(prefix, "\n ") } else prefix <- "Error : " msg <- paste0, conditionMessage(e), "\n") .Internal(seterrmessage(msg[1L])) if (!silent && ietOption("show.error.messages"))) { cat(msg, file = outFile) .Internal(ferredWarnings()) } invisible(structure(msg, class = "try-error", condition = e 6: try({ obj <- H5Dread(h5dataset = h5dataset, h5spaceFile = h5spaceFile, eMem = h5spaceMem, compoundAsDataFrame = compoundAsDataFrame, drop = drop, .. 7: h5readDataset(h5dataset, index = index, start = start, stride = stride, blockk, count = count, compoundAsDataFrame = compoundAsDataFrame, drop = drop, ...) 8: h5read(file, "PM/peakChrom") 9: doTryCatch(return(expr), name, parentenv, handler) 10: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 11: tryCatchList(expr, classes, parentenv, handlers) 12: tryCatch(binChrom <- h5read(file, "PM/peakChrom"), error = function(e) { stop(Warning @addPmat: 'PM/peakChrom' not found in ", file))}) 13: readPeaks(x) 14: FUN(X[[i]], ...) 15: lapply(fileList, function(x) { readPeaks(x)}) 16: addPmatToSnap.default(obj = combined, num.cores = 1) 17: addPmatToSnap(obj = combined, num.cores = 1)

So it seems to be saying that it can't find the PM matrix. Is there a way to check if it was added without running this function?

Ryan

[r3fang]

Hi Ryan, can you update the snaptools to the latest version of you are using python 3

[r3fang]

You can try to add pm to existing snap file. If it failed with error ‘pm’ already exist, it mean the snap file already has cell by peak matrix

[r3fang]

Hi Ryan,

I have changed pmat and gmat function now, it does not require snaptools, instead, it's a new function called createPmat and createGmat which should solve this problem. If your prbolem has not been solved, can you test the new function?

[RZiffra]

Everything is working well for me now with the new updates, thanks!

[aminbensaad]

@RZiffra how did you manage to make it work?