NanoComp / meep

free finite-difference time-domain (FDTD) software for electromagnetic simulations
GNU General Public License v2.0
1.21k stars 618 forks source link

meep: bug: unknown src_time_id (missing registration?) when using mpa.FourierFields with yee_grid=False #2839

Open HIEROT opened 4 months ago

HIEROT commented 4 months ago

I have read PR #1959 and am aware of the possible causes of the issue is that a chunk need to meddle with other chunk's sources. It seems solved, however, in the above PR, but I met this error again when mpi parallelization enabled. After some investigation, I figured out that it has something to do with MULTIPLE FourierFields parameter, which means after uncommenting the code below, the error raises when placing the adjoint sources.

    obj_list = [
        mpa.FourierFields(
            sim,
            vol_trans,
            mp.Ex,
        ),
        # mpa.FourierFields(
        #     sim,
        #     vol_trans,
        #     mp.Ey,
        # ),
        # mpa.FourierFields(
        #     sim,
        #     vol_trans,
        #     mp.Ez,
        # ),
    ]

Note that this can happen even in the first FORWARD run, where to my knowledge, dft sources have yet to be placed, particulaly it throws error JUST before a run ends.

Starting forward run...
Past barrier
-----------
Initializing structure...
Splitting into 20 chunks by cost
time for choose_chunkdivision = 0.00228272 s
Working in 3D dimensions.
Computational cell is 4.56694 x 4.56694 x 3.67238 with resolution 21.2396
     block, center = (34.125,50.375,-0.662199)
          size (4.55,4.55,1.3244)
          axes (1,0,0), (0,1,0), (0,0,1)
          dielectric constant epsilon diagonal = (2.08514,2.08514,2.08514)
     block, center = (34.125,50.375,0.4)
          size (3.25,3.25,0.8)
          axes (1,0,0), (0,1,0), (0,0,1)
time for set_epsilon = 0.290253 s
time for set_conductivity = 0.00549031 s
time for set_conductivity = 0.00592261 s
time for set_conductivity = 0.00592912 s
time for set_conductivity = 0.00582448 s
time for set_conductivity = 0.00574685 s
time for set_conductivity = 0.00562099 s
-----------
on time step 250 (time=6.72772), 0.016028 s/step
on time step 709 (time=19.0798), 0.00871686 s/step
on time step 1165 (time=31.3512), 0.00878124 s/step
on time step 1618 (time=43.5418), 0.00883985 s/step
on time step 2069 (time=55.6786), 0.00887105 s/step
on time step 2521 (time=67.8423), 0.00885186 s/step
on time step 2974 (time=80.033), 0.00884248 s/step
on time step 3424 (time=92.1429), 0.00890106 s/step
on time step 3859 (time=103.849), 0.00920765 s/step
on time step 4309 (time=115.959), 0.0088949 s/step
on time step 4760 (time=128.096), 0.00887642 s/step
meep: bug: unknown src_time_id (missing registration?)
Abort(1) on node 1 (rank 1 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1
meep: bug: unknown src_time_id (missing registration?)
Abort(1) on node 2 (rank 2 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, 1) - process 2