gathering arrays to plot

[francispoulin]

I am using spectralDNS as a guide to learn how to solve PDEs using mpi4py-fft and having a problem with the plotting. In particular, I want to gather the distributed arrays, namely coordinates and a field in order to be able to plot it with matplotlib.

I have included below my current attempt it doesn't work. What is the correct way of doing this given how newDistArrays are defined?

`# Initialize arrays xx = np.zeros((N[0],N[1])) yy = np.zeros((N[0],N[1])) uu = np.zeros((N[0],N[1]))

Gather fields

xx = MPI.COMM_WORLD.gather(X[0], root=0) yy = MPI.COMM_WORLD.gather(X[1], root=0) uu = MPI.COMM_WORLD.gather(ur, root=0)

Plot

plt.figure(figsize=(16,10)) plt.pcolormesh(uu) plt.colorbar() plt.show() `

[mikaem]

Hi Francis

When running in parallel plotting is usually done in Paraview after the solver finishes. You use the IO with HDF5 or NetCDF4 to write distributed arrays to file and then open these files on Paraview. The distributed array in mpi4py-fft does not come with a mesh attached. For solving PDE's I recommend using shenfun that has much more capabilities in that direction, and it's using mpi4py-fft for handling the distributed arrays.

That being said, there is a helper function in mpi4py-fft that can be used to gather distributed arrays. See z.get below, it can get any slice, and the complete array, on rank 0

from mpi4py import MPI
from mpi4py_fft.distarray import DistArray
from mpi4py_fft.mpifft import PFFT
comm = MPI.COMM_WORLD
N = (16, 16, 16)
fft = PFFT(MPI.COMM_WORLD, N)
z = DistArray(N, dtype=float)
z[:] = comm.Get_rank()
z0 = z.get((slice(None), slice(None), slice(None)))
print(comm.Get_rank(), z0.shape)

[francispoulin]

Hello Mikael,

Thanks again. You raise a lot of good points and I will reply to each of them briefly.

1) I agree that Paraview and VisIt are great for viewing data and I do use them both. Unfortunately, their plots are not always as nice as matplotlib and harder to control, which is why I like matplotlib for some things.

2) Thanks for the code. I see the example you presented was for DistArray but am happy to say that it also works for newDistArray. Very nice.

3) Thanks also for suggesting shenfun. I had seen it before and don't know it very well but from what I've seen it looks great. I noticed in spectralDNS it is stated that the example in mpi4py-fft is not necessarily for high performance and if you want that then the reader is directed to shenfun. I have used other languages before, mainly Fenics and Firedrake, but am very interested to work through those examples as well.

My reason for not going directly to that is a matter of taste. Shenfun uses spectral Galerkin approach and the method that I ideally want to use is pseudo-spectral. Both should work and a comparison might be fun. It seems to me that mpi4py-fft has enough machinery there to do what I need, after I figure out how to learn it. Then, it would be neat to compare with what shenfun can do and if the latter outperforms it then I will certainly change over.

In particular, I see that the eigenvalue problems are quite complicated, which is great, and I think i could build on this quite a bit on my research, with help from my students.

This issue's page is probably not ideal for this sort of discussion but I am happy to switch to email at any point if you like.

Cheers, Francis