search

marbl / merqury

k-mer based assembly evaluation
Other
272 stars 19 forks source link

Limit threads #36

Open StevenBai97 opened 3 years ago

StevenBai97 commented 3 years ago

Hi, How to limit threads when running merqury (merqury.sh)? I used default parameter and it seemed to take up too many cores. I'll really appreciate your suggestions. Thanks.

faguil commented 3 years ago

Hi,

I am facing the same issue here. At the moment, I have no access to a computer with high cores and memory RAM.

I think the issue arises in the spectra-cn plots because this script uses meryl to select the kmers greater than the selected kmer. I thought to edit the script and include the options threads and memory in order to fit my computer.

Is that OK?

Another question, is it possible to re-run an analysis? merqury is able to restart a previous run?

Regards,

brianwalenz commented 3 years ago

Environment variable OMP_NUM_THREADS= will limit CPU usage to cores.

There is no equivalent setting for memory usage, but you can (ab)use variables used by grid schedulers to achieve the same effect. SLURM_MEM_PER_NODE= will limit meryl to MEGABYTES of memory.

arangrhie commented 3 years ago

@faguil you could, depending on which stage you want to re start. There are some files that you may have to manually remove before restarting though in case the restart is due to a failure in a particular step.

In case that was in the very early process of the spectra-cn step, I'd recommend to do a fresh restart in a new dir.