number of cpus//threads

[EricDeveaud]

Hello

megahit uses multiprocessing.cpu_count() to ge the max value of cpus//threads. but multiprocessing.cpu_count() is the number of cpu in the machine. But this is not the same as the number of cpu available to the process. For example, you can run in a taskset context or a batch scheduler like slurm.

see:

$ nproc
96
$ taskset -c 1 nproc
1
$ taskset -c 1 python3 -c "import multiprocessing; print(multiprocessing.cpu_count())"
96

I would suggest to use len(os.sched_getaffinity(0)) instead of multiprocessing.cpu_count()

$ python3 -c "import os; print(len(os.sched_getaffinity(0)))"
96
$ taskset -c 1 python3 -c "import os; print(len(os.sched_getaffinity(0)))"
1

regards

Eric

[EricDeveaud]

Hello

sorry I just noticed that I did not pushthe fix for. this problem. see: https://github.com/voutcn/megahit/pull/349

regards

Eric

[aquaskyline]

Merged