search

BinPro / CONCOCT

Clustering cONtigs with COverage and ComposiTion
Other
119 stars 48 forks source link

CONCOCT uses more CPUs than requested with the --threads argument, unless user also sets OMP_THREAD_LIMIT to the right value #292

Open fpusan opened 3 years ago

fpusan commented 3 years ago

Hi team! Thanks for the awesome software (and for still maintaining it!).

As per the title, concoct uses more CPUs than instructed.

If I got your code correctly, thread usage is limited in the lines attached below,

https://github.com/BinPro/CONCOCT/blob/823dcd670bc42f6ea4622881c2484cda6c253a76/c-concoct/c_vbgmm_fit.c#L80-L81

but apparently this does not work as intended: even if the print message reports the right number of threads, CPU usage is actually higher.

CPU usage can be limited by setting the maximum number of threads via the OMP_THREAD_LIMIT environmental variable, but ideally the CONCOCT script should handle it without further user intervention.

Best regards,

Fernando

EDIT: Adding for reference: 1) Link to a discussion in our repository in which a user identified the bug (https://github.com/jtamames/SqueezeMeta/issues/69). 2) Link to a stackoverflow post discussing the workaround (https://stackoverflow.com/questions/37391492/openmp-creates-too-many-threads).

sannareddyk commented 1 year ago

Hi team, I'm running in to similar issue with the --threads argument, CONCOCT uses more CPUs than instructed. Has this issue been fixed or is there a work around for it?

Thanks for your help. Keerthi