*** caught segfault *** address 0xa14d9000, cause 'memory not mapped'

[EdBreen]

I'm new to STITCH but I'm having segfault problems. For example see error output below. The command issued was:

STITCH.R --chr=scaffold_9_RaGOO --bamlist=./TBA_EX46-HAC-Mod1-P.R2.RaGOO.PlusCpMt.sorted.dedup.bamlist --posfile=scaffold_9_RaGOO.posfile --outputdir=./ --K=450 --nGen=3 --nCores=8

caught segfault address 0xa14d9000, cause 'memory not mapped'

Traceback: 1: cpp_read_reassign(ord = ord, qnameInteger_ord = qnameInteger_ord, bxtagInteger_ord = bxtagInteger_ord, bxtag_bad_ord = bxtag_bad_ord, qname = qname, bxtag = bxtag, strand = strand, sampleReadsRaw = sampleReadsRaw, readStart_ord = readStart_ord, readEnd_ord = readEnd_ord, readStart = readStart, readEnd = readEnd, iSizeUpperLimit = iSizeUpperLimit, bxTagUpperLimit = bxTagUpperLimit, use_bx_tag = use_bx_tag, save_sampleReadsInfo = save_sampleReadsInfo) 2: merge_reads_from_sampleReadsRaw(sampleReadsRaw = sampleReadsRaw, qname = qname, bxtag = bxtag, strand = strand, readStart = readStart, readEnd = readEnd, iSizeUpperLimit = iSizeUpperLimit, use_bx_tag = use_bx_tag, bxTagUpperLimit = bxTagUpperLimit, save_sampleReadsInfo = save_sampleReadsInfo, qname_all = qname_all, readStart_all = readStart_all, readEnd_all = readEnd_all) 3: loadBamAndConvert(iBam = iBam, L = L, pos = pos, nSNPs = nSNPs, bam_files = bam_files, cram_files = cram_files, reference = reference, iSizeUpperLimit = iSizeUpperLimit, bqFilter = bqFilter, chr = chr, N = N, downsampleToCov = downsampleToCov, sampleNames = sampleNames, inputdir = inputdir, useSoftClippedBases = useSoftClippedBases, regionName = regionName, tempdir = tempdir, chrStart = chrStart, chrEnd = chrEnd, chrLength = chrLength, save_sampleReadsInfo = save_sampleReadsInfo, use_bx_tag = use_bx_tag, bxTagUpperLimit = bxTagUpperLimit) 4: FUN(X[[i]], ...) 5: lapply(X = S, FUN = FUN, ...) 6: doTryCatch(return(expr), name, parentenv, handler) 7: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 8: tryCatchList(expr, classes, parentenv, handlers) 9: tryCatch(expr, error = function(e) { call <- conditionCall(e) if (!is.null(call)) { if (identical(call[[1L]], quote(doTryCatch))) call <- sys.call(-4L) dcall <- deparse(call)[1L] prefix <- paste("Error in", dcall, ": ") LONG <- 75L sm <- strsplit(conditionMessage(e), "\n")[[1L]] w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w") if (is.na(w)) w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L], type = "b") if (w > LONG) prefix <- paste0(prefix, "\n ") } else prefix <- "Error : " msg <- paste0(prefix, conditionMessage(e), "\n") .Internal(seterrmessage(msg[1L])) if (!silent && isTRUE(getOption("show.error.messages"))) { cat(msg, file = outFile) .Internal(printDeferredWarnings()) } invisible(structure(msg, class = "try-error", condition = e))}) 10: try(lapply(X = S, FUN = FUN, ...), silent = TRUE) 11: sendMaster(try(lapply(X = S, FUN = FUN, ...), silent = TRUE)) 12: FUN(X[[i]], ...) 13: lapply(seq_len(cores), inner.do) 14: mclapply(1:length(sampleRanges), mc.cores = nCores, FUN = loadBamAndConvert_across_a_range, sampleRanges = sampleRanges, bundling_info = bundling_info, L = L, pos = pos, nSNPs = nSNPs, bam_files = bam_files, cram_files = cram_files, reference = reference, iSizeUpperLimit = iSizeUpperLimit, bqFilter = bqFilter, chr = chr, N = N, downsampleToCov = downsampleToCov, sampleNames = sampleNames, inputdir = inputdir, useSoftClippedBases = useSoftClippedBases, regionName = regionName, tempdir = tempdir, chrStart = chrStart, chrEnd = chrEnd, chrLength = chrLength, save_s

[jelber2]

I think I had a similar error when installing STITCH in my normal environment. I couldn't figure out what dependency was causing the problem, so I think I solved that by using the conda install for 1.6.3 (see https://github.com/rwdavies/STITCH/issues/38)

[EdBreen]

I think I had a similar error when installing STITCH in my normal environment. I couldn't figure out what dependency was causing the problem, so I think I solved that by using the conda install for 1.6.3 (see #38)

Thanks for reply we will look into this

[rwdavies]

Hi, this issue is still live, right?

So K=450 in general might cause the RAM to blow up, I'd recommend starting with something lower like K=10 and then scaling up once it's working

But the error message with cpp_read_reassign suggests an error interpreting the BAM files. Which is weird, as that part of the code is well tested.

How many SNPs are in the posfile? Any chance you're running out of RAM? Can you run in a high-RAM environment to confirm? Or can you try running just a single BAM at a time, and maybe turn on --generateInputOnly, to just process the BAM and quit, over each BAM?

If it's not RAM, I'm guessing there's a bug in my BAM processing code, which is possible, though that code is pretty well tested. If that's the case, I'd appreciate it if you're able to build an MWE you can send me, and I can debug it.

Thanks, Robbie

[EdBreen]

Thanks Robbie,

My reply is late because other things came up plus we wanted to try a few things out. We tried K=10, K=100, and K =450. All three processes failed. However, the R seg-fault problem magically disappeared, but these processes still failed because they all had some sort of tmpfile problem.

[2020-10-27 15:16:09] downsample sample Gid122001 - 180099 of 2843376 reads removed [2020-10-27 15:17:10] downsample sample Gid127718 - 24612 of 1533288 reads removed [2020-10-27 15:17:13] Done generating inputs [2020-10-27 15:17:13] Copying files onto tempdir sh: /tmp/Rtmp1E3MQX/HLPXSCFPAS/files_to_transfer.txt: No such file or directory rsync: failed to open files-from file /tmp/Rtmp1E3MQX/HLPXSCFPAS/files_to_transfer.txt: No such file or directory rsync error: syntax or usage error (code 1) at main.c(1568) [client=3.1.2] [2020-10-27 15:17:13] Done copying files onto tempdir [2020-10-27 15:17:13] Generate allele count [2020-10-27 15:17:14] Error in readChar(con, 5L, useBytes = TRUE) : cannot open the connection

Error in check_mclapply_OK(out2) : An error occured during STITCH. The first such error is above Calls: STITCH -> buildAlleleCount -> check_mclapply_OK In addition: Warning messages: 1: In mclapply(1:length(sampleRanges), mc.cores = nCores, FUN = loadBamAndConvert_across_a_range, : scheduled core 7 did not deliver a result, all values of the job will be affected 2: In mclapply(sampleRanges, mc.cores = nCores, FUN = buildAlleleCount_subfunction, : all scheduled cores encountered errors in user code Execution halted

As we were imputing using 2775942 SNPs (for 1 chromosome), we broke the chromosome into smaller chunks (~5 Mbp), and this is working. Each chunk then outputs a VCf and we imagine that combining these afterwards should not be too hard.