Malloc error - OSX

[mattloose]

Running on a sample dataset on Mac OSX I get a malloc error. I will share the data with you to reproduce.

ont2cram -i fast5_pass/ -o PASSCRAM
Loading Fast5 from: 'fast5_pass'
100%|████████████████████████████████████████████████████████████████| 9/9 [02:40<00:00, 17.81s/it]
Writing CRAM to: 'PASSCRAM'
 33%|█████████████████████▎                                          | 3/9 [02:34<04:29, 44.92s/it]python(4752,0x1136f75c0) malloc: can't allocate region
*** mach_vm_map(size=18446744071562067968) failed (error code=3)
python(4752,0x1136f75c0) malloc: *** set a breakpoint in malloc_error_break to debug
python(4752,0x1136f75c0) malloc: can't allocate region
*** mach_vm_map(size=18446744071562067968) failed (error code=3)
python(4752,0x1136f75c0) malloc: *** set a breakpoint in malloc_error_break to debug
Segmentation fault: 11

[AlexanderVi]

Tried to repro the issue on Ubuntu 18.04.2 and got a segmentation fault.

Thread 1 "python3" received signal SIGSEGV, Segmentation fault.
0x00007fffdbd2e96b in RansEncPutSymbol (sym=0x7fffffffd210, pptr=<synthetic pointer>, r=<synthetic pointer>) at /tmp/pip-req-build-fi5w9qub/htslib/cram/rANS_byte.h:280

(gdb) backtrace
#0  0x00007fffdbd2e96b in RansEncPutSymbol (sym=0x7fffffffd210, pptr=<synthetic pointer>,
    r=<synthetic pointer>) at /tmp/pip-req-build-fi5w9qub/htslib/cram/rANS_byte.h:280
#1  rans_compress_O0 (in=0x7ffe6bc93010 "\301\277\363s\367", <incomplete sequence \337>, in_size=2720271619,
    out_size=out_size@entry=0x7fffffffd520) at htslib/cram/rANS_static.c:159
#2  0x00007fffdbd30319 in rans_compress (in=<optimized out>, in_size=<optimized out>,
    out_size=out_size@entry=0x7fffffffd520, order=order@entry=0) at htslib/cram/rANS_static.c:767
#3  0x00007fffdbd2336f in cram_compress_by_method (strat=0, level=0, method=RANS,
    out_size=<synthetic pointer>, in_size=<optimized out>, in=<optimized out>) at htslib/cram/cram_io.c:1063
#4  cram_compress_block (fd=fd@entry=0x3385b20, b=0x1d243ab0, metrics=0x315e810, method=method@entry=3090,
    level=level@entry=5) at htslib/cram/cram_io.c:1203
#5  0x00007fffdbd1b1f4 in cram_compress_slice (c=<optimized out>, s=0x11c6a90, fd=0x3385b20)
    at htslib/cram/cram_encode.c:873
#6  cram_encode_slice (s=0x11c6a90, h=0x1a80b20, c=0x1a808a0, fd=0x3385b20) at htslib/cram/cram_encode.c:1053
#7  cram_encode_container (fd=fd@entry=0x3385b20, c=c@entry=0x1a808a0) at htslib/cram/cram_encode.c:1642
#8  0x00007fffdbd27261 in cram_flush_container (fd=0x3385b20, c=0x1a808a0) at htslib/cram/cram_io.c:3117
#9  0x00007fffdbd1c295 in cram_next_container (b=0x14b19200, b=0x14b19200, fd=0x3385b20)
    at htslib/cram/cram_encode.c:2456
#10 cram_put_bam_seq (fd=0x3385b20, b=0x14b19200) at htslib/cram/cram_encode.c:3101
#11 0x00007fffdbcd5371 in sam_write1 (fp=<optimized out>, h=<optimized out>, b=<optimized out>)
    at htslib/sam.c:1624
#12 0x00007fffda2a7133 in __pyx_f_5pysam_17libcalignmentfile_13AlignmentFile_write (
    __pyx_v_self=__pyx_v_self@entry=0x7fffecc8ef78, __pyx_v_read=__pyx_v_read@entry=0x7fffecc88408,
    __pyx_skip_dispatch=__pyx_skip_dispatch@entry=1) at pysam/libcalignmentfile.c:20518
#13 0x00007fffda2a7cef in __pyx_pf_5pysam_17libcalignmentfile_13AlignmentFile_28write (
    __pyx_v_read=0x7fffecc88408, __pyx_v_self=0x7fffecc8ef78) at pysam/libcalignmentfile.c:20663
#14 __pyx_pw_5pysam_17libcalignmentfile_13AlignmentFile_29write (__pyx_v_self=0x7fffecc8ef78,
    __pyx_v_read=0x7fffecc88408) at pysam/libcalignmentfile.c:20643
#15 0x000000000050a8ec in ?? ()
#16 0x000000000050c549 in _PyEval_EvalFrameDefault ()
#17 0x00000000005081d5 in ?? ()
#18 0x000000000050a020 in ?? ()
#19 0x000000000050aa1d in ?? ()
#20 0x000000000050c549 in _PyEval_EvalFrameDefault ()
#21 0x0000000000509ce8 in ?? ()

[jkbonfield]

What version of htslib is that using? Given this is pysam I assume it's whatever is bundled internally inside pysam so 1.9 at a guess. There has been a lot of code changing since then.

It looks like it attempted to malloc -2^31 (-2147483648) which was then turned into a very large 64-bit number and failed. The rANS_static.c code looks to be using int for the input buffer size, so it'll have problems compressing data in the 2Gb ish range, but it's really not designed for such large blocks of data and I don't think it's likely you'll have hit such a limit, is it? If so - something is wrong!

It's also possible the failed malloc was inside pysam somewhere and not the CRAM code, but it didn't trigger a failure and so it passed an illegal buffer to htslib which then resulted in the crash. I'd really need to see more to know as the debug stack track isn't particularly forthcoming with pointer values.

[AlexanderVi]

Thanks for looking at this. Yes, htslib version (1.9) which is bundled with pysam. If I send you a crash dump - would that help?

[jkbonfield]

Possibly, but I haven't got many days left this year so it may be worth postponing initially incase pysam upgrade to 1.10 meanwhile as that would make debugging it easier.