suitesparse "libopenblas.so.0: cannot open" build errors

[dslate]

When I build Julia on my Ubuntu 12.04.1 LTS x86_64 system I get a bunch of messages like the following:

./qrdemo: error while loading shared libraries: libopenblas.so.0: cannot open shared object file: No such file or directory

My latest build was Julia Version 0.0.0+95634459.r0a64, Commit 0a641b1dac (2012-09-02 19:46:45). I did a simple "git clone git://github.com/JuliaLang/julia.git" into an empty directory followed by "cd julia" and "make".

openblas seems to build ok. Here are some of the messages I get: 2012-09-03 01:26:01 (2.15 MB/s) - `openblas-v0.2.3.tar.gz' saved [3236430/3236430] ... END OF TESTS OK.

OpenBLAS build complete.

OS ... Linux Architecture ... x86_64 BINARY ... 64bit C compiler ... GCC (command line : gcc) Fortran compiler ... GFORTRAN (command line : gfortran) Library Name ... libopenblas_penrynp-r0.2.3.a (Multi threaded; Max num-threads is 2)

Both libopenblas.so and its symlink libopenblas.so.0 are created:

lrwxrwxrwx 1 david david 14 Sep 3 01:41 ./usr/lib/libopenblas.so.0 -> libopenblas.so -rwxrwxr-x 1 david david 12976792 Sep 3 01:37 ./usr/lib/libopenblas.so

But tests run by the make report not being able to find the shared library.

I know there have been some issues regarding openblas, but I can't seem to find something relevant to what I am encountering. Any ideas on what is going wrong?

[dmbates]

Check the contents of $JULIA_SRC/usr/lib to see if you have libopenblas.so and a symbolic link to that file from libopenblas.so.0

[dslate]

Yes, libopenblas.so and libopenblas.so.0 are there, as I mentioned in my original post.

[dmbates]

I should have read your report more carefully.

[vtjnash]

Just to clarify: Are these messages that result in a failed build? Does Julia run afterwards or is the build halted at some later point? Can you include the final error message is it is failing?

On Tuesday, September 4, 2012, dslate wrote:

When I build Julia on my Ubuntu 12.04.1 LTS x86_64 system I get a bunch of messages like the following:

./qrdemo: error while loading shared libraries: libopenblas.so.0: cannot open shared object file: No such file or directory

My latest build was Julia Version 0.0.0+95634459.r0a64, Commit 0a641b1https://github.com/JuliaLang/julia/commit/0a641b1dac(2012-09-02 19:46:45). I did a simple "git clone git://github.com/JuliaLang/julia.git" into an empty directory followed by "cd julia" and "make".

openblas seems to build ok. Here are some of the messages I get: 2012-09-03 01:26:01 (2.15 MB/s) - `openblas-v0.2.3.tar.gz' saved [3236430/3236430] ... END OF TESTS OK.

OpenBLAS build complete.

OS ... Linux Architecture ... x86_64 BINARY ... 64bit C compiler ... GCC (command line : gcc) Fortran compiler ... GFORTRAN (command line : gfortran) Library Name ... libopenblas_penrynp-r0.2.3.a (Multi threaded; Max num-threads is 2)

Both libopenblas.so and its symlink libopenblas.so.0 are created:

lrwxrwxrwx 1 david david 14 Sep 3 01:41 ./usr/lib/libopenblas.so.0 -> libopenblas.so -rwxrwxr-x 1 david david 12976792 Sep 3 01:37 ./usr/lib/libopenblas.so

But tests run by the make report not being able to find the shared library.

I know there have been some issues regarding openblas, but I can't seem to find something relevant to what I am encountering. Any ideas on what is going wrong?

— Reply to this email directly or view it on GitHubhttps://github.com/JuliaLang/julia/issues/1251.

[dslate]

The build succeeds, and I am able to run Julia. In fact, I am using Julia successfully for a fairly large program, but it involves little or no linear algebra, so I don't know whether I am missing any functionality provided by openblas. Is there a simple program I can run that will tell me whether Julia is able to use openblas?

[timholy]

From the Linux command prompt, type "make testall" in the main Julia directory. That will tell you if you have any missing functionality.

[dslate]

"make testall" seems to work just fine:

Script started on Wed Sep 5 13:46:05 2012 1david@LC2430HD:~/julia/julia$ make testall PERL test/unicode/UTF-32BE.unicode PERL test/unicode/UTF-32LE.unicode PERL test/unicode/UTF-16BE.unicode PERL test/unicode/UTF-16LE.unicode PERL test/unicode/UTF-8.unicode JULIA test/all * all * core * numbers * strings * unicode * corelib * hashing * remote * arrayops * lapack * factorizations * fft * sparse * arpack * bitarray * random * special * functional * bigint * distributions * combinatorics * statistics * glpk * linprog * bigfloat * poly * file * Rmath * zlib * perf SUCCESS 1david@LC2430HD:~/julia/julia$ exit exit

BTW, the original build got 33 of those "error while loading shared libraries: libopenblas.so.0: cannot open shared object file: No such file or directory" messages.

[nolta]

This is a known suitesparse build error. Seems to be harmless.

[dslate]

If those "error" messages are harmless, then that's fine. Do they occur on all Julia builds, or only on certain systems?

[kmsquire]

I noticed this a while back but didn't report. Unless you specify otherwise, programs in the SuiteSparse package are linked against libopenblas.so.0 in $JULIA/usr/lib. The full location is specified during linking, but on linux at least, the OS also needs to know about it when running the programs, and it only knows to look in the standard locations (/lib, /usr/lib, etc.) and locations that are specified in LD_LIBRARY_PATH (and maybe one other place I'm forgetting).

$ pwd
/home/kmsquire/src/julia/deps/SuiteSparse-4.0.2/SPQR/Demo
$ ./qrdemo < ../Matrix/a2.mtx
./qrdemo: error while loading shared libraries: libopenblas.so.0: cannot open shared object file: No such file or directory
$ LD_LIBRARY_PATH=/home/kmsquire/Source/julia/usr/lib ./qrdemo < ../Matrix/a2.mtx
Matrix      2-by-2      nnz:      3
residual:  0.0e+00 rank:      2
residual:  0.0e+00 rank:      2
residual:  0.0e+00 rank:      2

On OSX, I think julia is typically linked to the system blas libraries, so the error wouldn't occur there. On other systems, it depends on the setting of USE_SYSTEM_BLAS in Make.inc. One fix would be to temporarily add $JULIA/usr/lib to LD_LIBRARY_PATH.

Or, since the tests are not required for proper functioning of the library, it could be ignored.

[StefanKarpinski]

I think we now link against OpenBLAS on OS X too. It's faster than the system BLAS.

[dslate]

I know about LD_LIBRARY_PATH, having occasionally in the past set it in some (non-Julia-related) situations. But since Linux is a very common platform, should the SuiteSparse tests be "fixed" to set LD_LIBRARY_PATH appropriately?

[vtjnash]

I've just pushed a fix for this on mac and linux by adding the appropriate rpath to the build arguments. Note that on Mac this won't completely take effect until you've rebuilt all of deps, but it shouldn't be generally necessary to rebuild -- unless you want to have a build without errors in extraneous demo code (or you are installing, which may be broken if you don't completely rebuild). -jameson

On Sep 5, 2012, at 7:48 PM, dslate wrote:

I know about LD_LIBRARY_PATH, having occasionally in the past set it in some (non-Julia-related) situations. But since Linux is a very common platform, should the SuiteSparse tests be "fixed" to set LD_LIBRARY_PATH appropriately?

— Reply to this email directly or view it on GitHub.

[nolta]

@vtjnash's patch appears to work, so i'll go ahead and close.

[xianyi]

Hi @vtjnash ,

I just noticed this issue. Should I set rpatch in OpenBLAS ?

Xianyi

[vtjnash]

No, I think when you are installing to a known fixed location, setting the absolute path (as you already are doing) is probably better. Julia is kind-of a special case where we have a known rpath in the calling binaries, but not a known or fixed installation location.