search

SystemsGenetics / KINC

Knowledge Independent Network Construction
MIT License
11 stars 4 forks source link

mpirun of KINC slower on single machine than running serial vesion #116

Closed spficklin closed 4 years ago

spficklin commented 4 years ago

If I use mpirun -np 4 on a KINC command it ends up being very much slower than just using one CPU in serial mode. It will launch 4 instances but those instances only consume 8-9% of the CPU.

bentsherman commented 4 years ago

@spficklin The multicore performance depends a lot on how you run it. My guess is that your input data is too small to be split up evenly between the various processes. Also keep in mind that the first process is the master and doesn't do any work so with 4 cores you should see up to 3x speedup not 4x.

spficklin commented 4 years ago

Well, it took 6 hours to complete the task with just one CPU. I would have thought MPI would have allowed me to decrease that time.

bentsherman commented 4 years ago

It should give you speedup even with only 4 cores, but I need to know more about how you ran it to figure why it's not giving any speedup. Like how large is your input file and what command-line parameters are you using. Also you can try to run KINC several times at 1, 2, 3, ... cores, up to however many you have, and see how the runtime changes as you increase the number of cores. I can usually achieve near-linear speedup.

spficklin commented 4 years ago

Well, I just ran another test and with the current version of KINC, I'm not experiencing this slowness. I'm doing a run now the same dataset I used before and it's telling me 2 hours with 4 CPUs (rather than 6 hours with 1). So, given that I can't reproduce this problem I'm going to close it out.

spficklin commented 4 years ago

Oh, and I do see all 4 CPUs with 100% utilization. I'm not sure what went wrong before....