search

je-santos / MPLBM-UT

Library for performing multiphase simulations (based on the Shan-Chen model) in complicated geometries (i.e. porous media 3D images)
GNU General Public License v3.0
147 stars 57 forks source link

Problem with MPI #35

Closed qrfan closed 3 years ago

qrfan commented 3 years ago

Hi, Thanks for sharing such a great tool！This is very helpful for my research. I encountered a problem when using MPI. The use of MPI multi-core parallel computing does not increase the computing speed. The calculation time for one step is about 2 minutes, no matter how many cores are used, it is almost the same calculation efficiency. Is this normal? Is there anything that needs special attention in the parallel computing of the program? I tried OpenMPI 2.1.1 and MPICH 3, and they both have the same result. Looking forward to your reply. Thank you very much.

je-santos commented 3 years ago

Hi @qrfan ,

In my tests the code scales linearly with number of processors, so it looks like you are running it on a single one. Have you check your hardware utilization using htop/top ?

qrfan commented 3 years ago

I have checked it by using "top". I used "mpirun -np x ../../src/2-phase_LBM/ShanChen input_spherepack.xml". The number of the occupied core is "x". I did two tests.

  1. I tested some "showcases" in palabos, and it has obvious calculation acceleration when "x>1".
  2. I tested MPLBM-UT on my PC and a supercomputing center. The calculation time for one step is almost the same for the different core numbers.

How can I test out what the problem is?

je-santos commented 3 years ago

It seems that you are running the program n times, instead of running it in parallel. These problems are typically very particular with the local mpi installations and the modules that are loaded. You can try debugging it by running the code using srun or ibrun. Unfortunately, I'm far from a parallel programing expert. I'm sorry I can't be of more help.

qrfan commented 3 years ago

I asked the parallel technicians of the server to help with the test. The program runs in parallel, but the computing efficiency of multi-core is basically the same as that of one core. This situation is very strange because palabos and other software have parallel effects on the server. Have you tested the parallel efficiency of this version of code in " https://github.com/je-santos/MPLBM-UT "? We tested this case "https://github.com/je-santos/MPLBM-UT/tree/master/examples/unsteady_relperm_spherepack". And what are the compilation command and MPI software and version you used?

je-santos commented 3 years ago

Hi @qrfan ,

My desktop uses 1.4.1 (Hydra build) and the supercomputer uses 4.0.2 (OepenRTE).

Hope this helps