Problem With Data in Far-Fields Output to .h5 File

[dkeathley]

I have had a persistent issue using the Simulation.output_farfields() function. Whenever I put this at the end of my script and run it using mpirun, I see that the data saved is essentially random numerical values (a lot of very large numbers, and NaNs throughout).

I know that it is not an issue with my implementation as I have tested it using the Simulation.get_farfields() function with identical inputs (all else identical) and then used my own python code to then save the returned arrays to an h5 file using h5py.

It appears to me that the h5 file might be initialized correctly, but somehow the data is not written to the initialized file before the code exits.

I have tested this now on three machines, two running fedora and one running under windows subsystem for linux, all with identical results.

Happy to provide more details if needed.

Below is the relevant portion of code, with the commented out region at the bottom giving basically random data while the active code returns the far-fields correctly.

    #Create a near-field box for transforming to a far-field region
    nearfield_box = sim.add_near2far(fcen, df, nfreq_ff,
                                     mp.Near2FarRegion(center=mp.Vector3(0,0,0.5*sz), size=mp.Vector3(sx,sy,0), weight=+1),
                                     mp.Near2FarRegion(center=mp.Vector3(0,0,-0.5*sz), size=mp.Vector3(sx,sy,0), weight=-1),
                                     mp.Near2FarRegion(center=mp.Vector3(0,0.5*sy,0), size=mp.Vector3(sx,0,sz), weight=+1),
                                     mp.Near2FarRegion(center=mp.Vector3(0,-0.5*sy,0), size=mp.Vector3(sx,0,sz), weight=-1),
                                     mp.Near2FarRegion(center=mp.Vector3(0.5*sx,0,0), size=mp.Vector3(0,sy,sz), weight=+1),
                                     mp.Near2FarRegion(center=mp.Vector3(-0.5*sx,0,0), size=mp.Vector3(0,sy,sz), weight=-1))

    #Save the frequencies simulated for later reference during analysis
    far_fields_file = h5py.File(save_prefix + '-far-fields.h5', 'w', driver='mpio', comm=MPI.COMM_WORLD)
    freqs_ff = mp.get_near2far_freqs(nearfield_box)
    far_fields_file.create_dataset('freqs_ff', data=freqs_ff)

    # -- Run Simulation -- 
    sim.run(mp.at_beginning(mp.output_epsilon),
            mp.in_volume(near_field_monitor,
                         mp.to_appended("E_nf",
                                        mp.at_every(1/(fcen + df/2.0)/sampling_rate, mp.output_efield))),
            until_after_sources=mp.stop_when_fields_decayed(10,mp.Ey,mp.Vector3(0, 0, sz*0.5),1e-3))

    # -- Extract Far Fields -- 
    far_fields = sim.get_farfields(nearfield_box,
                           res_ff,
                           center=mp.Vector3(0, 0, z_ff),
                           size=mp.Vector3(width_ff, width_ff, 0))

    #Write fields out to the far_fields file:
    far_fields_file.create_dataset('ex_r', data=np.real(far_fields['Ex']))
    far_fields_file.create_dataset('ex_i', data=np.imag(far_fields['Ex']))
    far_fields_file.create_dataset('ey_r', data=np.real(far_fields['Ey']))
    far_fields_file.create_dataset('ey_i', data=np.imag(far_fields['Ey']))
    far_fields_file.create_dataset('ez_r', data=np.real(far_fields['Ez']))
    far_fields_file.create_dataset('ez_i', data=np.imag(far_fields['Ez']))

    far_fields_file.create_dataset('hx_r', data=np.real(far_fields['Hx']))
    far_fields_file.create_dataset('hx_i', data=np.imag(far_fields['Hx']))
    far_fields_file.create_dataset('hy_r', data=np.real(far_fields['Hy']))
    far_fields_file.create_dataset('hy_i', data=np.imag(far_fields['Hy']))
    far_fields_file.create_dataset('hz_r', data=np.real(far_fields['Hz']))
    far_fields_file.create_dataset('hz_i', data=np.imag(far_fields['Hz']))

    far_fields_file.close()

    # #Extract the far-fields at the desired region and save to .h5 file
    # sim.output_farfields(nearfield_box,
    #                      '_far_field',
    #                      res_ff,
    #                      center=mp.Vector3(0, 0, z_ff),
    #                      size=mp.Vector3(width_ff, width_ff, 0))

[oskooi]

Perhaps there is something wrong with your HDF5 library installation or you have multiple conflicting versions installed? If you built Meep from source, did you verify that the make check suite passes? If you installed the Conda package, you can try running e.g. test_cavity_farfield.py which only involves reading an HDF5 file (no writing).

Unfortunately, unlike get_farfields which is part of the test suite (test_n2f_periodic.py, test_cavity_farfield.py), output_farfields is not and so has not been properly tested (even though it is used in a tutorial example). There are C++ unit tests which involve writing and reading HDF5 files (i.e., [bragg_transmission.cpp]()) which is why it would be important to first establish that your HDF5 libraries are working correctly before investigating whether there is a bug in output_farfields.

[m9631m]

Hi @oskooi, thanks for your really great work on MEEP. I think this might be an actual bug. I did try it on multiple system with MEEP installed from conda and all had the same issue. To test I added these lines to binary_grating_n2f.ipynb

sim.output_farfields(
    n2f_obj,
    'test',
    ff_res,
    center=mp.Vector3(ff_distance, 0.5 * ff_length),
    size=mp.Vector3(y=ff_length),
)

It is the same arguments in the 'ff_supercell = sim.get_farfield' earlier in the notebook, but I only added a filename 'test' which sim.output_farfield requires.

Then I plotted the results in test.h5.

import h5py
with h5py.File('test.h5', 'r') as ff:
    plt.plot(ff['ez.i'])`

The content of test.h5 are basically some random float number. I had the same issue in my own code as well.

[evolphotonics]

Dear @oskooi, unfortunately, I have to agree with @m9631m. I cannot reproduce the results from the scheme tutorial on a M1 Mac (make checks passed fine). The saved .h5 fields have meaningless numbers (either exponentially high or low or nans)...Appreciate if you could pls investigate the scheme module output-farfields as well.