fftw on linux

[kilir]

Hi, I'm using the MIST plugin from the Fiji update site. Trying to use FFTW, the imageJ log file gives me Unable to load FFTW: java.lang.UnsatisfiedLinkError: gov.nist.isg.mist.lib32.imagetile.fftw.FFTW3Library32.fftwf_cleanup()V

These are the relevant parameters (I hope)

programType: FFTW
numCPUThreads: 12
loadFFTWPlan: false
saveFFTWPlan: false
fftwPlanType: MEASURE
fftwLibraryName: fftw3
fftwLibraryFilename: libfftw3.so
planPath: /home/rk/lib/fftw/fftPlans
fftwLibraryPath: /usr/lib

and I'm running 64 bit linux, and have fftw in /usr/lib

$ls /usr/lib/libfftw3*
/usr/lib/libfftw3f_mpi.so             /usr/lib/libfftw3l_omp.so             /usr/lib/libfftw3q_omp.so
/usr/lib/libfftw3f_mpi.so.3           /usr/lib/libfftw3l_omp.so.3           /usr/lib/libfftw3q_omp.so.3
/usr/lib/libfftw3f_mpi.so.3.6.10      /usr/lib/libfftw3l_omp.so.3.6.10      /usr/lib/libfftw3q_omp.so.3.6.10
/usr/lib/libfftw3f_omp.so             /usr/lib/libfftw3l.so                 /usr/lib/libfftw3q.so
/usr/lib/libfftw3f_omp.so.3           /usr/lib/libfftw3l.so.3               /usr/lib/libfftw3q.so.3
/usr/lib/libfftw3f_omp.so.3.6.10      /usr/lib/libfftw3l.so.3.6.10          /usr/lib/libfftw3q.so.3.6.10
/usr/lib/libfftw3f.so                 /usr/lib/libfftw3l_threads.so         /usr/lib/libfftw3q_threads.so
/usr/lib/libfftw3f.so.3               /usr/lib/libfftw3l_threads.so.3       /usr/lib/libfftw3q_threads.so.3
/usr/lib/libfftw3f.so.3.6.10          /usr/lib/libfftw3l_threads.so.3.6.10  /usr/lib/libfftw3q_threads.so.3.6.10
/usr/lib/libfftw3f_threads.so         /usr/lib/libfftw3_mpi.so              /usr/lib/libfftw3.so
/usr/lib/libfftw3f_threads.so.3       /usr/lib/libfftw3_mpi.so.3            /usr/lib/libfftw3.so.3
/usr/lib/libfftw3f_threads.so.3.6.10  /usr/lib/libfftw3_mpi.so.3.6.10       /usr/lib/libfftw3.so.3.6.10
/usr/lib/libfftw3l_mpi.so             /usr/lib/libfftw3_omp.so              /usr/lib/libfftw3_threads.so
/usr/lib/libfftw3l_mpi.so.3           /usr/lib/libfftw3_omp.so.3            /usr/lib/libfftw3_threads.so.3
/usr/lib/libfftw3l_mpi.so.3.6.10      /usr/lib/libfftw3_omp.so.3.6.10       /usr/lib/libfftw3_threads.so.3.6.10

Since it seems to complain about the 32bit version, I installed this as well

$ls /usr/lib32/libfftw3*
/usr/lib32/libfftw3f_omp.so             /usr/lib32/libfftw3l.so                 /usr/lib32/libfftw3q.so
/usr/lib32/libfftw3f_omp.so.3           /usr/lib32/libfftw3l.so.3               /usr/lib32/libfftw3q.so.3
/usr/lib32/libfftw3f_omp.so.3.6.10      /usr/lib32/libfftw3l.so.3.6.10          /usr/lib32/libfftw3q.so.3.6.10
/usr/lib32/libfftw3f.so                 /usr/lib32/libfftw3l_threads.so         /usr/lib32/libfftw3q_threads.so
/usr/lib32/libfftw3f.so.3               /usr/lib32/libfftw3l_threads.so.3       /usr/lib32/libfftw3q_threads.so.3
/usr/lib32/libfftw3f.so.3.6.10          /usr/lib32/libfftw3l_threads.so.3.6.10  /usr/lib32/libfftw3q_threads.so.3.6.10
/usr/lib32/libfftw3f_threads.so         /usr/lib32/libfftw3_omp.so              /usr/lib32/libfftw3.so
/usr/lib32/libfftw3f_threads.so.3       /usr/lib32/libfftw3_omp.so.3            /usr/lib32/libfftw3.so.3
/usr/lib32/libfftw3f_threads.so.3.6.10  /usr/lib32/libfftw3_omp.so.3.6.10       /usr/lib32/libfftw3.so.3.6.10
/usr/lib32/libfftw3l_omp.so             /usr/lib32/libfftw3q_omp.so             /usr/lib32/libfftw3_threads.so
/usr/lib32/libfftw3l_omp.so.3           /usr/lib32/libfftw3q_omp.so.3           /usr/lib32/libfftw3_threads.so.3
/usr/lib32/libfftw3l_omp.so.3.6.10      /usr/lib32/libfftw3q_omp.so.3.6.10      /usr/lib32/libfftw3_threads.so.3.6.10

I tried calling different libraries (e.g. libfftw3f) as well as the 32bit verions (while I was wondering why it does so since there's also src/main/java/gov/nist/isg/mist/lib/imagetile/fftw/FFTW3Library.java ) but to no success.

I guess I might be simply missing something and any hint would be highly appreciated.

Cheers, Rüdiger

[tblattner]

Hello Rüdiger,

I believe you will want to try to specify: fftwLibraryFilename: libfftw3.so

as libfftw3f.so

Same with: fftwLibraryName: fftw3, should be fftw3f

This should use the single precision version for FFTW. That should explain why we are seeing issue loading FFTW3Library32.fftwf_cleanup()

Regards, Tim

[kilir]

Hi Tim, thanks for the quick reply. So I tried

fftwLibraryName: fftw3f
fftwLibraryFilename: libfftw3f.so

and neither the the 64bit nor the 32bit version helps. Error message remains the same.

I'm using fftw 3.3.10, might this be the issue?

Cheers, Rüdiger

[tblattner]

Hi Rüdiger,

Checking the symbols in libfftw3f.so could verify if that function is defined, just to see if 3.3.10 changes anything. I suspect it should be the same. Something like nm -D libfftw3f.so | grep fftwf_cleanup

Are you using the GUI in MIST? On Windows I specify "FFTW Library File" to be: C:\Users\username\Downloads\fiji-win64\Fiji.app\lib\fftw\libfftw3f.dll

I'll see if I can setup my Linux environment to test that version of FFTW and installed in a similar location to see if I can reproduce the error. I actually will be on leave starting tomorrow, so I won't be able to get to it until next week.

In the meantime here is the relevant code that we use for initializing the library:

  /**
   * Initializes native libraries
   *
   * @param libFFTWPath     the path to the FFTW library
   * @param libUtilFnsPath  the path to the UtilFns library
   * @param fftwLibraryName the name of the FFTW library
   * @return true if all libraries are loaded, otherwise false
   */
  public static boolean initLibrary(String libFFTWPath, String libUtilFnsPath,
                                    String fftwLibraryName) {
    boolean loaded = false;
    try {
      BridJ.addNativeLibraryAlias("fftwf", fftwLibraryName);
      BridJ.addNativeLibraryAlias("fftwf", "fftwf3");
      BridJ.addNativeLibraryAlias("fftwf", "fftwf3-3");
      BridJ.addLibraryPath(libFFTWPath);
      FFTW3Library32.fftwf_cleanup();
      Log.msg(LogType.INFO, "FFTW library loaded successfully");
      loaded = true;

    } catch (UnsatisfiedLinkError ex) {
      Log.msg(LogType.MANDATORY, "Unable to load FFTW: " + ex.toString());
      return false;
    } catch (Exception e) {
      Log.msg(LogType.MANDATORY, "Unable to load FFTW: " + e.toString());
      return false;
    }

In there we try to add various aliases to use when loading the library, then add the library path after that. Checking for the existance of fftwf_cleanup in your library would probably be the next step to make sure it exists. If it does exist, then we need to find out why BridJ is not finding it...

Regards, Tim

[kilir]

Hi Tim, yes, I use the gui in imageJ.

nm -D /usr/lib/libfftw3f.so | grep fftwf_cleanup
00000000001298f0 T fftwf_cleanup

suggest that 3.3.10 should be fine.

I manually compiled fftw and installed it to /usr/local/lib but BridJ again returns the error.

libfftw3f is a link to libfftw3f.so -> libfftw3f.so.3.6.10 but also specifying the library libfftw3f.so.3.6.10 does not yield any improvement.

Just for testing I also manually compiled nfft 3.3.10 on an old Mac and there it works nicely using fftw3f - unfortunately this isn't the machine I'd like to use for performance reasons.

I'll see if I can make sense from the initialization code.

Cheers, Rüdiger

[tblattner]

This is a massive long-shot, but looking at the documentation that we wrote several years ago, we did mention this: Linux: libfftw3.so (default installation location "/usr/local/lib")

So we could try putting the library into /usr/local/lib? Please keep me posted on your tests! We'll sort this issue out one way or another...

[kilir]

Hi Tim, I'd say it's unrelated to the path, since other functions seems to be found. Interestingly, on Debian, everything works fine (no matter where the lib is located as long as it is properly specified), however on a default Arch Linux install the issue is quite reproducible, in both cases using Fiji and Fiji_nojre - which is unfortunately the System in question.

Cheers, Rüdiger

[kilir]

Ok, I think I found the culprit which seems to be the Fiji I use since some years and constantly updated using the updater, using a "fresh" install works like a charm, so I guess it must have been something rather Java than fftw-related.

Thank you for the effort and support!

Cheers, Rüdiger