Save and import FFT wisdom

[zywang-cosmo]

Hi, I am wondering if PencilFFTs has the function to export and import the FFT wisdom file, such as FFTW.import_wisdom() and FFTW.export_wisdom() in FFTW.jl? Thx!

[jipolanco]

Hi, no, that's currently not possible.

In principle it should be possible to implement such a feature. In practice I'd need to think about the best way to implement this.

Note that each MPI process will produce a different FFTW wisdom file for each 1D transform, meaning that the total number of wisdom files to save and load is 2 * d * nproc where d is the number of dimensions (the factor 2 is to account for forwards and backwards transforms). Maybe it's possible to only keep the wisdom file associated to one process, but since in general the number of non-transformed dimensions can vary from one process to the other, I'm not sure if this would actually work (but I'm not that familiar with wisdom files, so I might be wrong).

[erny123]

Is there a possibility of just saving the plan into a file then reloading it? So that if the plan's pencil dimensions matches the new pencil array then it could be used for that?

something like:

t_transform = (Transforms.FFT!(), Transforms.NoTransform!())
tplan = PencilFFTPlan(pen, t_transform; fftw_flags = FFTW.MEASURE,)

fid = open(PHDF5Driver(),"PencilTransformsPlan.hdf5", comm; write=true);
fid["1DFT"] = tplan;

[erny123]

So the only issue with using HDF5 is that most of the PencilArray types are not supported for some reason.

There has to be a definition in the PHDF5Driver I'm guessing? If anyone knows where to start point me to it. I'll start looking into it.

[jipolanco]

Using the HDF5 format is a good idea. However I'm not sure we would gain much by using the PHDF5Driver from PencilArrays, since its purpose is to write PencilArrays in parallel, and such arrays are not contained in PencilFFTPlan objects.

So the simplest would be to write all the fields contained in a PencilFFTPlan, possibly excluding temporary data buffers (plan.ibuf, plan.obuf) and timing information (plan.timer). Note that this also includes 1D FFTW plans, which vary from one process to another, but in principle if the pencil dimension is preserved then this is not a problem (we just write one plan per process). The question of how to "convert" FFTW plans to HDF5 and read it back is less clear to me. In short, all of this should be possible, but does require some work.

[erny123]

@jipolanco makes sense. I'll take a look at it since I think it would definitely be helpful. Can you give me some resources on how you wrote up the PHDF5 driver? I'm a little confused and can't find good documentation for Julia PHDF5 drivers

Thanks again for for this package! Definitely been extremely useful.

Edit: Just saw your reply for this: https://github.com/jipolanco/PencilArrays.jl/issues/90 I'd be more than happy to take a shot at it. I'll probably have some time over the weekend to start.

[jipolanco]

That's great if you want to take a shot at this!

If it's of any help, the PHDF5Driver for PencilArrays is defined in this file. This section of the HDF5.jl docs can also be useful.