search

FFTW / fftw3

DO NOT CHECK OUT THESE FILES FROM GITHUB UNLESS YOU KNOW WHAT YOU ARE DOING. (See below.)
GNU General Public License v2.0
2.75k stars 666 forks source link

is mpi wisdom cumulative? #33

Open chichilalescu opened 9 years ago

chichilalescu commented 9 years ago

I'm trying to save MPI wisdom, and I'm getting strange behavior. Since my understanding is that fftw wisdom accumulates, I figured the easiest thing I can do is create an empty file, then read wisdom from there, run my code, save wisdom in the file, and then the next time I would run my code, it would no longer spend any time with plans. However, that's not what happens.

In my tests I am using an MPI code on my 8 core machine, and I am doing an inverse FFT for 3 interleaved 256x256x256 arrays. After the first run with "FFTW_MEASURE" the wisdom file is big (64 lines), and it takes ~100 seconds for this first run. Second run takes 5 seconds, and the wisdom file is smaller (18 lines). Third run takes ~100 seconds, and the wisdom file remains the same (I guess, I didn't check all the codes individually). All subsequent runs are of ~100 seconds.

If I change my code so that it no longer overwrites the wisdom file after the first run, all subsequent runs are fast, so I can work around this issue. Also, for this particular size of the transform, FFTW_ESTIMATE seems to be just as fast as FFTW_MEASURE (if I don't overwrite the wisdom file), so it's not a big deal anyway. However, when I'll be running my production jobs, the transforms will be a lot bigger, and I'd like to understand what's happening.

By the way, I'm using the recommended way of saving the files from http://www.fftw.org/doc/FFTW-MPI-Wisdom.html (more on that in the next issue though).

chichilalescu commented 9 years ago

PS: I think I'm using FFTW 3.3.4, but the contents of the wisdom file start with "fftw-3.3.3". Is it possible I messed up my installation? I just installed it with "configure/make/make install", and I can only find one instance of each library file, so my guess is that the string is hardcoded somewhere and was not updated.