Odd dataset memory errors

[AndrewSkelton]

Hi,

Thanks for the host of tools developed :)

We've had some issues where Alevin-Fry (ran with the SimpleAF wrapper) where we experience a strange memory error. The command has worked successfully for a number of samples and just wonder if there's anything going wrong in your hands? Noteworthy that they run fine with CellRanger. SimpleAF version 0.14.1.

ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR268/030/SRR26865530/SRR26865530_1.fastq.gz
ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR268/030/SRR26865530/SRR26865530_2.fastq.gz
ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR268/028/SRR26865528/SRR26865528_1.fastq.gz
ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR268/028/SRR26865528/SRR26865528_2.fastq.gz

simpleaf quant \
    --reads1 ${sample_R1} \
    --reads2 ${sample_R2} \
    --threads 2 \
    --index Index_SimpleAF_GRCh38/index \
    --chemistry 10xv2 --resolution cr-like \
    --expected-ori both --unfiltered-pl \
    --t2g-map Index_SimpleAF_GRCh38/index/t2g_3col.tsv \
    --output ./${sample}_v2_GRCh38

Error:

Error: piscem mapping failed with exit status ExitStatus(unix_wait_status(11)

Any insight would be very welcome :)

[rob-p]

Hi @AndrewSkelton,

Thanks for the report! We'd be happy to look into this. However, first I'd want to ask (and I kind of hate to be this dev ;P) can you try with the latest version? Specifically, with the latest version of piscem (which would be pulled in if you grab the latest version of simpleaf if you're using conda, or which you can install independently if you want to update just piscem)? The relevant piscem versions should have minimal-to-no effect on specific mapping results, but solves a few memory related bugs (undefined behavior) that could lead to failure in some odd corner cases. This is, in part, because piscem relies on piscem-cpp in its core, which is currently still written in C++.

We have plans to replace the whole core with Rust code which would hopefully minimize or eliminate such issues in the future, but we've not gotten around to that yet, so for the time being, we're addressing such cases as they arise. It could be that the issue you see here is one that triggers a bug we've already resolved. Otherwise, we're happy to look into it and fix it!

Best, Rob

[AndrewSkelton]

I thought that might be the response :p - I'll rebuild our container and give it a shot, shouldn't take much time at all.

[AndrewSkelton]

@rob-p that did the trick, latest version ran to completion. I've never encountered this problem before and ran it on a good number of varied samples... any intuition on what was odd about these samples to drive memory related undefined behaviours? Feel free to close, thanks for the speedy response and help!

[rob-p]

Hi @AndrewSkelton --- good to hear! Between the version you were using before and what is in the latest release, there were several fixes for rare corner cases, but one that jumps out was a potential null pointer dereference that was checked and fixed one or two releases ago. While alevin-fry is all in Rust, the core mapping code of piscem relies on sshash and is still in C++, so we are still every once in a while tracking down and fixing memory errors. Luckily @RagnarGrootKoerkamp is working on a Rust rewrite of sshash which will let us move the entire codebase over to Rust, hopefully minimizing or eliminating these corner cases (that get progressively rarer, and therefore harder to trigger and track down)!