Unable to use flint.so on juliabox VM instance

[wbhart]

Recently you kindly installed flint2, mpir and mpfr for me on the juliabox VM.

Unfortunately, something is still preventing me from actually using those libraries from within Julia on JuliaBox.

Assuming "/usr/local/lib" has been push!'ed onto DL_LOAD_PATH, I should be able to do the following from within the Nemo package (just Pkg.clone("https://github.com/wbhart/Nemo.jl") to get Nemo):

using Nemo.Rings
using Nemo.Fields
R, x = PolynomialRing(ZZ, "x")

But this doesn't work.

I have checked that I can compile and run a program which uses flint from the console in JuliaBox, using gcc. So the libraries themselves seem valid. But when trying to access symbols in flint from Julia itself, I get, for example:

a = ZZ(1)
error compiling ZZ: could not load module libflint: libflint: cannot open shared object file: No such file or directory
while loading In[4], in expression starting on line 1

My past experience is that this is caused by Julia being able to find flint, but the system not being able to resolve dependencies of libflint.so.

1) flint2 depends on mpir and mpfr (there is no direct dependency on the later from Nemo)

2) Some versions of GCC add some dependencies, which aren't always installed on systems, or can't be found by the linker, specifically libgcc_s sometimes ends up being required by flint.

3) pthreads can be a dependency, and sometimes it doesn't know where to find the pthreads that flint was linked against.

I honestly don't understand why it works when Nemo builds flint and its dependencies itself, but not when they are prebuilt and installed in the VM.

Could it be possible that either MPIR or flint was built without specifying the locations of MPIR or MPFR to configure?

It's puzzling, because Nemo works fine on JuliaBox when it builds its dependencies itself.

By the way, M4 is only a dependency of the MPIR build, not an actual runtime dependency.

[staticfloat]

There are two ways you might be able to resolve this.

• Manually dlopen all necessary dependencies for libflint before opening libflint proper. This will use DL_LOAD_PATH to search for the dependencies, which the system linker does not do.
• Append the necessary paths to LD_LIBRARY_PATH, which is the variable that the system linker (and thus Julia) pays attention to. This is slightly more dangerous exactly because it modifies the behavior of ALL child processes of Julia, but sometimes that's what you need. In this case, you want to modify the behavior of the system linker as it opens up libflint.

[wbhart]

I tried appending the paths to LD_LIBRARY_PATH. I don't know if this affects Julia unless it is done before Julia is actually started and from the same shell.

Manually dlopen'ing the libraries also didn't seem to work. I can dlopen each of libgmp.so, libmpfr.so and libflint.so, (which fails if the dependencies aren't opened, so we know this part is working). But still, Julia cannot use symbols from flint. That is extremely puzzling.

Can I check, was mpir built with the --enable-gmpcompat option? If not, mpfr and flint will have built itself against the gmp.so which was presumably already there. Though I can't see why this would be an issue, as flint always builds against libgmp.so nowadays, whether an actual GMP or a gmpcompat'ed MPIR.

Is the VM image cross-compiled? On the other hand, as everything works fine when linked via gcc, I don't see that this can have caused problems.

[wbhart]

Interestingly enough I can also find symbols in libflint, after dlopening libgmp, libmpfr and libflint, using dlsym. It's just Julia's ccall that doesn't work. How is that even possible?

[wbhart]

OK, the combination that works is to dlopen libgmp.so, libmpfr.so, libflint.so and push!(DL_LOAD_PATH, "/usr/local/lib"). Then everything works as expected.

What we can do for now is check if the environment variable HOSTNAME is set to "juliabox". If so, we can execute the additional steps above.

[ViralBShah]

Hopefully, this tides it over for now. Going forward, we should do something better.

[tanmaykm]

Yes, mpir was built with --enable-gmpcompat. The options used for building libraries can be seen in this file: https://github.com/JuliaLang/JuliaBox/blob/master/docker/IJulia/Dockerfile

Strangely, ENV["HOSTNAME"] did not seem to work inside Julia when I tried. But gethostname() did.

[tanmaykm]

Ah, HOSTNAME is not exported from the shell. gethostname() would probably be better in any case.

[wbhart]

Thanks, I'll switch to that.

On 14 January 2015 at 08:11, Tanmay Mohapatra notifications@github.com wrote:

Ah, HOSTNAME is not exported from the shell. gethostname() would probably be better in any case.

— Reply to this email directly or view it on GitHub https://github.com/JuliaLang/JuliaBox/issues/190#issuecomment-69877512.

[tanmaykm]

Closing this as resolved. Please reopen if anything further is required.