UPHL-BioNGS / Cecret

Reference-based consensus creation
MIT License
49 stars 26 forks source link

Issue setting maxcpus with v.2.0.2021115 #70

Closed keckstrom closed 2 years ago

keckstrom commented 2 years ago

I just updated to the newest version but I'm having issues with the cpu settings. I've been running the previous versions for a few months now (thanks for your hard work on this!) without issue on a machine with 6 cores + Docker, but now I'm getting errors such as: Error executing process > 'bwa '

Caused by: Process requirement exceed available CPUs -- req: 8; avail: 6

I've tried to override the params.maxcpus & params.medcpus and even hard coding -t or --threads to 6 throughout the Cecret.nf file for each process, but so far the error persists and the message starting the run displays: The maximum number of CPUS used in this workflow is 8

Any ideas on what I should try next?

erinyoung commented 2 years ago

@keckstrom

I think this issue may be coming up more.

Cecret used to calculate cpu usage automatically depending on what system that it was launched from, but this was causing havoc with those that used cloud computer devices, so it was changed to a fixed value. This is still how Cecret can still operate if you have cloned the github repository and don't use the -profile option.

Something like the following (you can adjust the paths to what is on your system)

nextflow run Cecret.nf -c configs/singularity.config

How have you "overriding" the params.maxcpus and params.medcpus?

Something to the effect of

nextflow run UPHL-BioNGS/Cecret -profile singularity --maxcpus 6

should adjust the maxcpus for your needs.

keckstrom commented 2 years ago

I was trying to set it in the config file before, but didn't think anything of it with the change to using profiles instead. This works perfectly, thanks!

erinyoung commented 2 years ago

I'm glad it's now working for you!

erinyoung commented 2 years ago

I added in some documentation about this issue so that others don't run into problems in the future.

Thank you for brining this to my attention!