search

jtamames / SqueezeMeta

A complete pipeline for metagenomic analysis
GNU General Public License v3.0
357 stars 78 forks source link

How to specify memory and core parameter in command line? #651

Closed kittyshady closed 1 year ago

kittyshady commented 1 year ago

Hi there,

I am very new to Squeezemeta and bioinformatics in general. I checked that I have ~126 Gb of memory and 32 core.

I ran perl SqueezeMeta.pl -m coassembly -p tuanao_run -s samples.file -f 1-rename --cleaning -a spades -c 2000 -map bowtie -binners maxbin, metabat, concoct

and saw that it only used ~6Gb of the memory I have and only 5 core. What parameters do I use to make it use all of the resources I have? I tried to add/change -b with multiple numbers but looked like it still doesn't increase the memory usage.

PS. I have 9 metagenomes and I am into 6 hours of trimmomatic (currently at sample 7).

Thanks!

fpusan commented 1 year ago

-t 32 for specifying the number of threads. Memory management should happen on its own, no need to touch that unless you were being short in memory

kittyshady commented 1 year ago

Thanks! All the cores seem to be used now, but only 4 cores run at 100% while others use only 5-10%. Is this normal? Is there a way to make every core run at 100%?

fpusan commented 1 year ago

That's normal. E.g. if you are doing lots of read/write operations, speed will be limited by how fast can you access the data in the disk (this changes depending on whether you have hdd, ssd, network drives, etc...), rather than by how fast your CPUs are. In that case some cores will be idle sometimes since they are waiting to get the data they need.