frequent segfaults

[jvolkening]

I am getting frequent segfaults when testing out FaStore. They occur both on release 0.8 binaries and on the latest master commit built using GCC 4.9.2. The exact errors depend on the input file and the compression settings. Some examples:

Compressing a file with 100 reads:

$ ./fastore_compress.sh --reduced --in 100.fq --out comp_test --threads 24
./fastore_compress.sh: line 171:  5576 Segmentation fault      $FASTORE_PACK e "-i$TMP_REBIN-8" "-o$OUT_PACK" "-t$TH_PACK" $PAR_PACK_C1 $PAR_PE -v

Compressing a file with 100k reads:

$ ./fastore_compress.sh --reduced --in 100k.fq --out comp_test --threads 24
./fastore_compress.sh: line 171:  5410 Segmentation fault      $FASTORE_BIN e "-i$IN_FQ" "-o$TMP_BIN" "-t$TH_BIN" $PAR_ID $PAR_QUA $PAR_BIN_C1 $PAR_PE

Compressing a file with 100k reads, lossless compression, single thread:

$ ./fastore_compress.sh --lossless --in ../100k.fq --out C8 --threads 1
./fastore_compress.sh: line 171:  6140 Segmentation fault      $FASTORE_REBIN e "-i$TMP_REBIN-2" "-o$TMP_REBIN-4" "-t$TH_REBIN" $PAR_REBIN_C1 $PAR_PE -p4

Compressing a file with only four reads succeeds. Compressing the 100-read file using --lossless succeeds. These reads are all subsampled from the same dataset. Compressing the test files included in the release tarball also works. The errors have occurred on multiple systems, but all are running Debian 9 x84_64 with GCC 4.9.2. The software doesn't appear to be running out of memory (or anywhere even close).

Let me know if there is more information I can provide.

Thanks, Jeremy

[lrog]

Hi Jeremy,

thanks for your detailed report! I'll take a look into the issue. Could you only please provide some sample file on which FaStore crashes?

Thanks, Lucas

[jvolkening]

A sample file is here:

https://gist.github.com/jvolkening/2b7d6e67845df9bc285af6b2e4cedd77

../scripts/fastore_compress.sh --in ~/scratch/test_1000.fq --out FOO --reduced

throws

../scripts/fastore_compress.sh: line 227: 14026 Segmentation fault      $FASTORE_REBIN e "-i$TMP_BIN" "-o$TMP_REBIN-2" "-t$TH_REBIN" $PAR_REBIN_C1 $PAR_PE -p2

[lrog]

Thanks @jvolkening ! I'll revisit this issue while working on #4 as the problem can be related.

[lrog]

Hi @jvolkening Sorry for quite delay with the possible fix. I've just merged the fix to master. Could you possibly check it whether it solves your issue?

[jvolkening]

This seems to have fixed the earlier segfaults I was seeing. However, I'm still running into some problems. I will open a separate issue for them.