MTJ-related loading warnings on Windows and Linux

[claczny]

I get the following message when running VizBin (revision01) on a Windows or Linux machine (w/ Java 7):

2014-12-01 08:39:08,619 DEBUG [Thread-0] (ProcessInput.java:88) - Running PCA... (Mtj)
Dec 01, 2014 8:39:09 AM com.github.fommil.netlib.LAPACK <clinit>
WARNING: Failed to load implementation from: com.github.fommil.netlib.NativeSystemLAPACK
Dec 01, 2014 8:39:09 AM com.github.fommil.netlib.LAPACK <clinit>
WARNING: Failed to load implementation from: com.github.fommil.netlib.NativeRefLAPACK
2014-12-01 08:39:14,801 DEBUG [Thread-0] (DataSetUtils.java:249) - DONE: Computed the new basis.
2014-12-01 08:39:15,127 DEBUG [Thread-0] (DataSetUtils.java:256) - DONE: Projected from sample to eigen space.

The program returns a visualization, so apparently something is running and creating the initial projection. However, this behavior is not optimal.

I also came across the following posts but neither of them provided a clear solution for me, for now. https://github.com/fommil/matrix-toolkits-java/issues/38 https://github.com/fommil/matrix-toolkits-java/issues/50

[claczny]

This seems to be expected behavior: Quoting from the README of netlib-java:

In netlib-java, implementations of BLAS/LAPACK/ARPACK are provided by:

    delegating builds that use machine optimised system libraries (see below)
    self-contained native builds using the reference Fortran from netlib.org
    F2J to ensure full portability on the JVM

The JNILoader will attempt to load the implementations in this order automatically.

and it provides directions on how to include the Linux and Windows libraries. The last point (about F2J) seems to be the reason why it works on Linux/Windows despite apparently missing the necessary native code. Accordingly, this represents a nice fallback strategy but most likely at the price of performance.

Accordingly, this appears to require some effort from our side. The Linux libraries might be relatively easy to resolve (building the MTJ-<version>.jar in our Minimal Build Environment) but if this works that easily too with the Windows libraries is not clear. Although we have a cross-compilation environment running...

UPDATE 1: Following the instruction in the netlib-java README, I ran sudo apt-get install libatlas3-base libopenblas-base to install the necessary dynamically-loaded libraries on Linux. Running VizBin after that, I got the following message:

2014-12-01 09:39:52,760 DEBUG [Thread-0] (ProcessInput.java:88) - Running PCA... (Mtj)
Dec 01, 2014 9:39:52 AM com.github.fommil.jni.JniLoader liberalLoad
INFO: successfully loaded /tmp/jniloader8103932422613978857netlib-native_system-linux-x86_64.so
2014-12-01 09:39:53,431 DEBUG [Thread-0] (DataSetUtils.java:249) - DONE: Computed the new basis.
2014-12-01 09:39:53,710 DEBUG [Thread-0] (DataSetUtils.java:256) - DONE: Projected from sample to eigen space.

suggesting a successful loading and also an extremely fast projection. The installation of LAPACK under Windows does not seem to be that straight-forward, but we will have to check (e.g., http://webserver2.tecgraf.puc-rio.br/~ismael/Cursos/YC++/downloads/Lapack/docs/Lapack_Windows_install.html).

[claczny]

Installing liblapack.dll, libblas.dll, and liblapacke.dll in C:\Windows\System32 fixed the com.github.fommil.netlib.NativeRefLAPACK related warning but not the com.github.fommil.netlib.NativeSystemLAPACKrelated warning. It is somewhat unexpectd though that it says "successfully loaded".

2014-12-02 14:41:05,770 DEBUG [Thread-3] (ProcessInput.java:88) - Running PCA... (Mtj)
Dez 02, 2014 2:41:06 PM com.github.fommil.netlib.LAPACK <clinit>
Warnung: Failed to load implementation from: com.github.fommil.netlib.NativeSystemLAPACK
Dez 02, 2014 2:41:07 PM com.github.fommil.jni.JniLoader liberalLoad Information: successfully loaded C:\Users\Jennifer\AppData\Local\Temp\jniloader2636391227429958662netlib-native_ref-win-i686.dll
2014-12-02 14:41:09,654 DEBUG [Thread-3] (DataSetUtils.java:249) - DONE: Computed the new basis.
2014-12-02 14:41:10,637 DEBUG [Thread-3] (DataSetUtils.java:256) - DONE: Projected from sample to eigen space.

[claczny]

Changing the logging level to "finest" might be a way to get more information from the JNILoader. However, I don't know yet how to specify this at runtime. Maybe this post helps...