NanoComp / meep

free finite-difference time-domain (FDTD) software for electromagnetic simulations
GNU General Public License v2.0
1.17k stars 598 forks source link

openmp+mpi hangs on farfield transform #2638

Closed Deutschland12 closed 10 months ago

Deutschland12 commented 10 months ago

I have compiled meep from source with mpi and openmp support. When trying to run a sim involving a Near2Far transform, the program successfully performs a get_farfield() for a single point, but hangs when trying to perform get_farfields() for a square region, UNLESS I only use a single mpi process (mpirun -np 1 python foo.py). Changing the number of mpi processes does speed up the geometry and field propagation steps as expected.

The build also failed a few of the unit tests (run with make RUNCODE="env OMP_NUM_THREADS=4 mpirun -np 3" check) including test_binary_grating.py and get_epsilon_grid.py

EDIT: in case it's relevant, the build was performed inside a conda environment (I do not have root access, but the necessary packages have been installed in root as suggested in https://meep.readthedocs.io/en/latest/Build_From_Source/#meep) EDIT 2: changing the value of OMP_NUM_THREADS also does not seem to change the speed of the N2F steps (when using 1 mpi process so it doesn't hang)