search

marbl / meryl

A genomic k-mer counter (and sequence utility) with nice features.
113 stars 13 forks source link

meryl terminates with 'std::bad_alloc' and failed assertion #35

Open wangjie07070910 opened 1 year ago

wangjie07070910 commented 1 year ago

Hi! I am encountering an error when running meryl, I used 20 CPU threads, 150GB of memory. It seems to be failing on the assertion _blockPosition <= merylutil::ftell(F). Also, before this assertion failure, it throws a 'std::bad_alloc' exception. This usually happens when there isn't enough memory to allocate, but I've specified 150GB of memory in my job submission script, which I believe should be sufficient. Here is the relevant information:

terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc meryl: utility/src/kmers-v1/kmers-files.H:137: void merylutil::kmers::v1::merylFileIndex::set(merylutil::kmers::v1::kmpref, FILE, uint64): Assertion `_blockPosition <= merylutil::ftell(F)' failed. meryl: utility/src/kmers-v1/kmers-files.H:137: void merylutil::kmers::v1::merylFileIndex::set(merylutil::kmers::v1::kmpref, FILE, uint64): Assertion `_blockPosition <= merylutil::ftell(F)' failed.

Failed with 'Aborted'; backtrace (libbacktrace): (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)() (null)::0 in (null)()

brianwalenz commented 1 year ago

Try reducing the number of threads to 16 or even 8. I can't advise more without knowing the command line.

wangjie07070910 commented 1 year ago

Many thanks for your response. Here you go: meryl threads=20 memory=150 k=21 count output 138880_test.meryl 138880.clean_1.fq 138880.clean_2.fq Besides, I've put the thread down to 8 as you suggested, I'll update you if it reports an error again.

wangjie07070910 commented 1 year ago

meryl-count-male.err.txt I put the thread down to 8 and still failed to solve it. Attached is the full log file. Thanks in advance!