crash on empty vcf input

[wheaton5]

Hey Ian, Pat, and whoever else is looking at this. I'm using vartrix and haven't had a problem until I tried it on some human data and I'm splitting on chromosome regions without chunking small chromosomes (such as alt haplotype chroms) together. So I first call variants in those regions with freebayes --pooled-continuous to get candidate variant sites. So many of these chunks have no candidate variants but proper headers. I will get around this by just not calling vartrix on these, but I think for robustness it would be good to not crash on this.

one of my tomls is as follows

name = 'vartrix' operating_system = 'unix:Ubuntu' crate_version = '1.1.0' explanation = ''' Panic occurred in file 'libcore/slice/mod.rs' at line 690 ''' method = 'Panic' backtrace = ''' stack backtrace: 0: 0x2b90f78b8f5c - backtrace::backtrace::libunwind::trace::habb00f14bd19719d at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.9/src/backtrace/libunwind.rs:53

• backtrace::backtrace::trace::h723ba52d9872a98e at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.9/src/backtrace/mod.rs:42 1: 0x2b90f78b7f32 - backtrace::capture::Backtrace::new_unresolved::ha78f2aff003d75b3 at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.9/src/capture.rs:88
• backtrace::capture::Backtrace::new::h75a7ede5ecccbd9a at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.9/src/capture.rs:63
• ::default::hb511d64f96ad4085 at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.9/src/capture.rs:235 2: 0x2b90f78b7fa8 - backtrace::capture::Backtrace::new::h75a7ede5ecccbd9a 3: 0x2b90f78a97dc - human_panic::report::Report::new::he0638e2d5d38d9d7 at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/human-panic-1.0.1/src/report.rs:45 4: 0x2b90f78a8fc2 - human_panic::handle_dump::h7f7e81f73680ebff at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/human-panic-1.0.1/src/lib.rs:197 5: 0x2b90f787c77a - vartrix::main::{{closure}}::h706516ea68ad695f at /mnt/yard2/ian/test_vartrix/vartrix/:18 6: 0x2b90f7a0ddf3 - std::panicking::rust_panic_with_hook::he4c3a67f6258a8f9 at libstd/panicking.rs:515 7: 0x2b90f7a0db49 - std::panicking::continue_panic_fmt::h156c04b2aea348c7 at libstd/panicking.rs:426 8: 0x2b90f7a0d865 - rust_begin_unwind at libstd/panicking.rs:337 9: 0x2b90f7a660cb - core::panicking::panic_fmt::h0e6d5c6fb2a34dd6 at libcore/panicking.rs:92 10: 0x2b90f7a65ffa - core::panicking::panic::hfbb77505dc622acd at libcore/panicking.rs:53 11: 0x2b90f7875d1f - core::slice::::chunks::h7e1ecc7d07d73fcc at /checkout/src/libcore/macros.rs:20
• vartrix::_main::h24c2e877acc73eba at src/main.rs:205 12: 0x2b90f786ff12 - vartrix::main::hd5dddde4b39f4a62 at src/main.rs:143 13: 0x2b90f7880a02 - std::rt::lang_start::{{closure}}::hc97668c284d6a8cc at /checkout/src/libstd/rt.rs:74 14: 0x2b90f7a0d802 - std::rt::lang_start_internal::{{closure}}::h664d976655ba006f at libstd/rt.rs:59
• std::panicking::try::do_call::hac56dcc01f36ef70 at libstd/panicking.rs:310 15: 0x2b90f7a2e7d9 - __rust_maybe_catch_panic at libpanic_unwind/lib.rs:105 16: 0x2b90f7a08205 - std::panicking::try::hcda2c11533bcd35b at libstd/panicking.rs:289
• std::panic::catch_unwind::h28820092503d86ef at libstd/panic.rs:392
• std::rt::lang_start_internal::h249c3823b1cd2120 at libstd/rt.rs:58 17: 0x2b90f787c7f9 - main 18: 0x2b90f86317ec - __libc_start_main 19: 0x2b90f786c828 - '''

[ifiddes-10x-zz]

Good catch, I will fix this ASAP.

[ifiddes-10x-zz]

Would it be more desirable for VarTrix to crash with a human-understandable error at this point, write this to ERROR level log but still return a 0, or write to log and still produce empty matrix files?

For your use case, where I assume that you are automatically concatenating the resulting matrices, it seems like last option might be the best?

[wheaton5]

Interesting. I actually think that a human understandable error and returning 0 is preferable even though it would have made my life easier for it to have the empty matrix file. From the rust philosophy of frontloading errors it will prevent more issues downstream to outright error on "bad" input than to silently do the right thing with "bad" input. But it is debatable and you should make the call.

[ifiddes-10x-zz]

I agree. It is also much easier to implement the 'kill and report the error' version :)

[pmarks]

Hi Haynes, thanks for the report. I agree it's debatable... my initial reaction in cases like this is to try to make a null matrix. It's conceptually simpler, because it's the same 'type' of output, rather than a special case that downstream user needs to think about. Ian, does it seem like it will be hard to make an empty matrix? Does the mex format support it?

On Tue, Nov 20, 2018 at 10:59 AM ifiddes-10x notifications@github.com wrote:

I agree. It is also much easier to implement the 'kill and report the error' version :)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/10XGenomics/vartrix/issues/22#issuecomment-440391585, or mute the thread https://github.com/notifications/unsubscribe-auth/AACzA_paPVFd2Wa31ot_HfVFBueIlh1Wks5uxFEZgaJpZM4Yqwru .

[ifiddes-10x-zz]

It looks like the format supports it. It won't be too hard, just need to re-do some logic.

[wheaton5]

Hey guys. getting a few more errors. Figured I would keep it on this issue because it says its the same line number. But the vcf isn't empty this time. So far I just have 2 chunks failing with 1-2 variants per file. Haven't looked at the rust myself maybe it doesn't find cells supporting according to its criteria and it doesnt like outputting empty matrices?

toml here name = 'vartrix' operating_system = 'unix:Ubuntu' crate_version = '1.1.0' explanation = ''' Panic occurred in file 'libcore/slice/mod.rs' at line 690 ''' method = 'Panic' backtrace = ''' stack backtrace: 0: 0x2b5817e80f5c - backtrace::backtrace::libunwind::trace::habb00f14bd19719d at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.9/src/backtrace/libunwind.rs:53

• backtrace::backtrace::trace::h723ba52d9872a98e at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.9/src/backtrace/mod.rs:42 1: 0x2b5817e7ff32 - backtrace::capture::Backtrace::new_unresolved::ha78f2aff003d75b3 at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.9/src/capture.rs:88
• backtrace::capture::Backtrace::new::h75a7ede5ecccbd9a at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.9/src/capture.rs:63
• ::default::hb511d64f96ad4085 at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.9/src/capture.rs:235 2: 0x2b5817e7ffa8 - backtrace::capture::Backtrace::new::h75a7ede5ecccbd9a 3: 0x2b5817e717dc - human_panic::report::Report::new::he0638e2d5d38d9d7 at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/human-panic-1.0.1/src/report.rs:45 4: 0x2b5817e70fc2 - human_panic::handle_dump::h7f7e81f73680ebff at /mnt/home/ian.fiddes/.cargo/registry/src/github.com-1ecc6299db9ec823/human-panic-1.0.1/src/lib.rs:197 5: 0x2b5817e4477a - vartrix::main::{{closure}}::h706516ea68ad695f at /mnt/yard2/ian/test_vartrix/vartrix/:18 6: 0x2b5817fd5df3 - std::panicking::rust_panic_with_hook::he4c3a67f6258a8f9 at libstd/panicking.rs:515 7: 0x2b5817fd5b49 - std::panicking::continue_panic_fmt::h156c04b2aea348c7 at libstd/panicking.rs:426 8: 0x2b5817fd5865 - rust_begin_unwind at libstd/panicking.rs:337 9: 0x2b581802e0cb - core::panicking::panic_fmt::h0e6d5c6fb2a34dd6 at libcore/panicking.rs:92 10: 0x2b581802dffa - core::panicking::panic::hfbb77505dc622acd at libcore/panicking.rs:53 11: 0x2b5817e3dd1f - core::slice::::chunks::h7e1ecc7d07d73fcc at /checkout/src/libcore/macros.rs:20
• vartrix::_main::h24c2e877acc73eba at src/main.rs:205 12: 0x2b5817e37f12 - vartrix::main::hd5dddde4b39f4a62 at src/main.rs:143 13: 0x2b5817e48a02 - std::rt::lang_start::{{closure}}::hc97668c284d6a8cc at /checkout/src/libstd/rt.rs:74 14: 0x2b5817fd5802 - std::rt::lang_start_internal::{{closure}}::h664d976655ba006f at libstd/rt.rs:59
• std::panicking::try::do_call::hac56dcc01f36ef70 at libstd/panicking.rs:310 15: 0x2b5817ff67d9 - __rust_maybe_catch_panic at libpanic_unwind/lib.rs:105 16: 0x2b5817fd0205 - std::panicking::try::hcda2c11533bcd35b at libstd/panicking.rs:289
• std::panic::catch_unwind::h28820092503d86ef at libstd/panic.rs:392
• std::rt::lang_start_internal::h249c3823b1cd2120 at libstd/rt.rs:58 17: 0x2b5817e447f9 - main 18: 0x2b5818bf97ec - __libc_start_main 19: 0x2b5817e34828 - '''

[ifiddes-10x-zz]

Ah, it seems like it is the chunks method of Vec that is causing the problems. I need to make sure that the number of chunks is always non-zero. It can be zero when the number of threads exceeds the number of variants, or when the number of variants is 0. I just pushed a fix to the empty_vcf branch. If you are compiling it yourself, you can try that one out. Or wait for Travis to run then I can upload a new binary.

[ifiddes-10x-zz]

I have put up a pre-release binary for 1.1.1 that fixes this issue.