"Buffer overflow detected" when running HapCUT2

[olavurmortensen]

I've included two logs, one with the *** buffer overflow detected *** message, and just the log from HapCUT2.

I was using HapCUT2 with hash: fd01a1d2794c990d986a720a13905b167eb943d7

Don't know what else I can share to help debug, please let me know if there is something.

HapCUT2 log:

[2019:05:28 13:21:16] fragment file: linked_fragments
[2019:05:28 13:21:16] variantfile (VCF format):sample.vcf
[2019:05:28 13:21:16] haplotypes will be output to file: haplotypes
[2019:05:28 13:21:16] solution convergence cutoff: 5
[2019:05:28 13:21:16] QVoffset: 33

[2019:05:28 13:21:16] Calling Max-Likelihood-Cut based haplotype assembly algorithm
[2019:05:28 13:21:16] read 8303 variants from sample.vcf file 
[2019:05:28 13:21:17] no of non-trivial connected components 411 max-Degree 1372 connected variants 3752 coverage-per-variant 14.254797 
[2019:05:28 13:21:17] fragments 18367 snps 8303 component(blocks) 411
[2019:05:28 13:21:17] processed fragment file and variant file: fragments 18367 variants 8303
[2019:05:28 13:21:21] OUTPUTTING PRUNED HAPLOTYPE ASSEMBLY TO FILE haplotypes
[2019:05:28 13:21:21] OUTPUTTING PHASED VCF TO FILE haplotypes.phased.VCF
/mnt/fargen/experiments/joint_call/tests/phase_vcf/work/39/4ad289e1b932337dadbd4be4d8d6c7/.command.sh: line 2:  9566 Aborted                 (core dumped) /mnt/fargen/experiments/joint_call/tests/phase_vcf/HapCUT2/build/HAPCUT2 --nf 1 --outvcf 1 --fragments linked_fragments --VCF sample.vcf --output "haplotypes"

Error message:

*** buffer overflow detected ***: /mnt/fargen/experiments/joint_call/tests/phase_vcf/HapCUT2/build/HAPCUT2 terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f27fe1107e5]
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x5c)[0x7f27fe1b211c]
/lib/x86_64-linux-gnu/libc.so.6(+0x117120)[0x7f27fe1b0120]
/lib/x86_64-linux-gnu/libc.so.6(+0x116689)[0x7f27fe1af689]
/lib/x86_64-linux-gnu/libc.so.6(_IO_default_xsputn+0x80)[0x7f27fe1146b0]
/lib/x86_64-linux-gnu/libc.so.6(_IO_vfprintf+0x139b)[0x7f27fe0e750b]
/lib/x86_64-linux-gnu/libc.so.6(__vsprintf_chk+0x84)[0x7f27fe1af714]
/lib/x86_64-linux-gnu/libc.so.6(__sprintf_chk+0x7d)[0x7f27fe1af66d]
/mnt/fargen/experiments/joint_call/tests/phase_vcf/HapCUT2/build/HAPCUT2[0x409330]
/mnt/fargen/experiments/joint_call/tests/phase_vcf/HapCUT2/build/HAPCUT2[0x4122a0]
/mnt/fargen/experiments/joint_call/tests/phase_vcf/HapCUT2/build/HAPCUT2[0x4013d1]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f27fe0b9830]
/mnt/fargen/experiments/joint_call/tests/phase_vcf/HapCUT2/build/HAPCUT2[0x401a89]
======= Memory map: ========
00400000-00416000 r-xp 00000000 00:2c 17982507                           /mnt/fargen/experiments/joint_call/tests/phase_vcf/HapCUT2/build/HAPCUT2
00615000-00616000 r--p 00015000 00:2c 17982507                           /mnt/fargen/experiments/joint_call/tests/phase_vcf/HapCUT2/build/HAPCUT2
00616000-00617000 rw-p 00016000 00:2c 17982507                           /mnt/fargen/experiments/joint_call/tests/phase_vcf/HapCUT2/build/HAPCUT2
00617000-0061c000 rw-p 00000000 00:00 0 
01933000-0352e000 rw-p 00000000 00:00 0                                  [heap]
7f27fdc63000-7f27fdc79000 r-xp 00000000 fc:00 3146249                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f27fdc79000-7f27fde78000 ---p 00016000 fc:00 3146249                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f27fde78000-7f27fde79000 rw-p 00015000 fc:00 3146249                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f27fde98000-7f27fe099000 rw-p 00000000 00:00 0 
7f27fe099000-7f27fe259000 r-xp 00000000 fc:00 3155118                    /lib/x86_64-linux-gnu/libc-2.23.so
7f27fe259000-7f27fe459000 ---p 001c0000 fc:00 3155118                    /lib/x86_64-linux-gnu/libc-2.23.so
7f27fe459000-7f27fe45d000 r--p 001c0000 fc:00 3155118                    /lib/x86_64-linux-gnu/libc-2.23.so
7f27fe45d000-7f27fe45f000 rw-p 001c4000 fc:00 3155118                    /lib/x86_64-linux-gnu/libc-2.23.so
7f27fe45f000-7f27fe463000 rw-p 00000000 00:00 0 
7f27fe463000-7f27fe56b000 r-xp 00000000 fc:00 3155113                    /lib/x86_64-linux-gnu/libm-2.23.so
7f27fe56b000-7f27fe76a000 ---p 00108000 fc:00 3155113                    /lib/x86_64-linux-gnu/libm-2.23.so
7f27fe76a000-7f27fe76b000 r--p 00107000 fc:00 3155113                    /lib/x86_64-linux-gnu/libm-2.23.so
7f27fe76b000-7f27fe76c000 rw-p 00108000 fc:00 3155113                    /lib/x86_64-linux-gnu/libm-2.23.so
7f27fe76c000-7f27fe792000 r-xp 00000000 fc:00 3155096                    /lib/x86_64-linux-gnu/ld-2.23.so

[olavurmortensen]

This seems to become a problem when the number of variants is large. The example above is with 13 843 variants, but another example with 1419 variants works fine.

[vibansal]

Can you run with the option "--outvcf 0" to see if the error still occurs. This will confirm that the error is in the code (recently added) that outputs the phased VCF.

[olavurmortensen]

@vibansal I ran HapCUT2 with --outvcf 0 and got the error below instead, but no *** buffer overflow detected *** error. Looks like it isn't occurring when writing the VCF.

[2019:05:29 11:46:47] fragment file: linked_fragments
[2019:05:29 11:46:47] variantfile (VCF format):nocalls_removed.vcf
[2019:05:29 11:46:47] haplotypes will be output to file: haplotypes
[2019:05:29 11:46:47] solution convergence cutoff: 5
[2019:05:29 11:46:47] QVoffset: 33

[2019:05:29 11:46:47] Calling Max-Likelihood-Cut based haplotype assembly algorithm
[2019:05:29 11:46:47] read 13839 variants from nocalls_removed.vcf file 
[2019:05:29 11:46:47] no of non-trivial connected components 428 max-Degree 1024 connected variants 4088 coverage-per-variant 14.089041 
[2019:05:29 11:46:47] fragments 19672 snps 13839 component(blocks) 428
[2019:05:29 11:46:47] processed fragment file and variant file: fragments 19672 variants 13839
[2019:05:29 11:46:51] OUTPUTTING PRUNED HAPLOTYPE ASSEMBLY TO FILE haplotypes

[vibansal]

Would it be possible for you to share the fragment and VCF files?

[olavurmortensen]

Unfortunately no, this is confidential data.

[vibansal]

In that case, can you run HapCUT2 using the debugger gdb and share the output?

[olavurmortensen]

I can try, I have used gdb before. Can you tell me how, briefly?

[vibansal]

gdb -ex=r --args path_HAPCUT2_binary --fragments frags.txt --VCF input.vcf --out out.haps

If the program exits abnormally, run 'backtrace' on the (gdb) prompt and get the output.

You can share the output via email (vibansal at ucsd.edu) if needed.

[olavurmortensen]

Thanks. I've sent you the output. HapCUT2 executed in the same way as I wrote above, and I issued the backtrace command as you said.

[vibansal]

The problem was in the code that outputs the phased VCF file. I have pushed an update to the 'master' branch. Let me know if it works or not.

[olavurmortensen]

That fixed it, thanks a bunch!