search

jtamames / SqueezeMeta

A complete pipeline for metagenomic analysis
GNU General Public License v3.0
346 stars 81 forks source link

Died at line 276 in sqm_longreads.pl #759

Closed FloatingFloss closed 7 months ago

FloatingFloss commented 7 months ago

Hi,

I keep getting an error of "Died at /home/WUR/floss001/miniconda3/envs/SqueezeMeta/bin/sqm_longreads.pl line 276." when running "sqm_longreads.pl -p projectname -s equivfile -f /download/Dataset-1/UnzipGood/AllSeq -b 31" What is especially weird is that it only happens for about 20% of samples. Nevertheless, any ideas what is going wrong there? I am out of ideas, thank you.

fpusan commented 7 months ago

Are you running more than one sample in parallel? or other process that could be consuming RAM?

FloatingFloss commented 7 months ago

Thank you for your quick response . Yes, I am running an array of multiple in parallel. If I run only one it does not happen.

fpusan commented 7 months ago

You are just running out of memory. Run every process sequentially and it will work

FloatingFloss commented 7 months ago

For my understanding, I thought "- b" would make sure that every run would get a certain amount of memory. How am I running out of memory then? I have unfortunately a lot of samples to run (1000), running them sequentially will take quite a lot of time. So I would like to run them in parallel if possible.

fpusan commented 7 months ago

Yes -b does controls memory usage but it is also dataset dependent, so it is not surprising that it is failing in some cases. Running sequentially will actually be faster: you can anyways use the -t argument to use all the threads available to your system for DIAMOND mapping, and this way you can have a higher -b which also has speed benefits (as the reference database will be split into smaller chunks).