Reading mesh data in parallel

[krober10nd]

Hello,

Is there a tutorial or a short set of codes that can help me read my model parameter data into my solver when I am using several processors?

For example, in serial I have a file containing a model parameter defined at the vertices of the mesh. But when I run in parallel, my function f is sized by the number of DOFs I have on my processor. So the equality f.data.data[:] = inside segfaults the code.

I need the linear indices into the vector "inside" that correspond with the mesh vertices each processor owns.

        # Use the external data function to interpolate the values of f.
         f = Function(V)
         vp = open('VpOnNodes_Exact.txt', 'r')
         inside = vp.readlines()
         vp.close()
         f.dat.data[:] = inside

Thanks

[krober10nd]

It seems like I may just need to write my own interpolation function that each processor passes the mesh coordinates it owns for this to work.

[wence-]

Is this part of the docs helpful? https://www.firedrakeproject.org/interpolation.html#interpolation-from-external-data

[krober10nd]

Yep that helps. In order to do the interp onto the colocation points, I wrapped this in a function call.

https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.RegularGridInterpolator.html#scipy.interpolate.RegularGridInterpolator

For seismic data that I'm dealing with though, it will be impossible to load up a file on every core. Will have to get more clever.

Best

On Tue, Dec 10, 2019, 7:37 PM Lawrence Mitchell notifications@github.com wrote:

Is this part of the docs helpful? https://www.firedrakeproject.org/interpolation.html#interpolation-from-external-data

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/firedrakeproject/firedrake/issues/1564?email_source=notifications&email_token=AEOBZ7CP4DM2P5PSNY724ETQYAK3ZA5CNFSM4JZE45U2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGRGRQA#issuecomment-564291776, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEOBZ7DAMR56VKXZPNPODLTQYAK3ZANCNFSM4JZE45UQ .

[wence-]

For seismic data that I'm dealing with though, it will be impossible to load up a file on every core. Will have to get more clever.

Doing this right is somewhat more involved. We have some work in progress/starting to improve parallel interpolation (from external data and between non-matching meshes). If you're interested in helping out, or trying things as they appear, let us know!

[krober10nd]

Yes, I'm keen to help out with that and also try new things. My naive way of doing it would be to extract ranges from the NetCDF file containing the 3D data, but this doesn't use parallel file I/O.

In general, getting several model parameters on the meshes in 2 and 3D is a pain. I recall when I was heavily involved in coastal ocean modeling efforts using the FEM ADCIRC model, the biggest slow down was reading in high-frequency meteorology data into the model.

It would be nice to have a projection operator that can do this data management in parallel.

[dham]

Well it kind of could if you used pnetcdf, I think.