Building fails with undefined references to pthread_create, pthread_join, and main

[ianwilliamson]

I've successfully built MUMPS by executing make in the src/MUMPS_5.0.1/ subdirectory against the included libmetis as well as the system-wide metis. However, when I run make in src/ it finishes with the below error:

gfortran -fcray-pointer -O   mumps_cmplx_p.f90  mumps_p.f90 call_mumps_cmplx_p.f90 call_mumps_p.f90   -o ../lib/MUMPS -IMUMPS_5.0.1/include -LMUMPS_5.0.1/lib -ldmumps -lzmumps -lmumps_common -lpord  -L/usr/lib  -lmetis  -LMUMPS_5.0.1/libseq -lmpiseq  -lblas
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
MUMPS_5.0.1/lib/libmumps_common.a(mumps_io_thread.o): In function `mumps_low_level_init_ooc_c_th':
mumps_io_thread.c:(.text+0x28b): undefined reference to `pthread_create'
MUMPS_5.0.1/lib/libmumps_common.a(mumps_io_thread.o): In function `mumps_clean_io_data_c_th':
mumps_io_thread.c:(.text+0xed8): undefined reference to `pthread_join'
collect2: error: ld returned 1 exit status
Makefile:14: recipe for target 'all' failed
make: *** [all] Error 1

It looks like this is some issue with the included Fortran functions that are interfacing with Julia, but I'm not 100% sure. Any idea on how to get past this?

[rShekhtman]

In the gfortran command line, there must be a -shared option. It tells the compiler to create a library instead of an executable. Do yo know why it is missing?

[ianwilliamson]

Thanks very much for that suggestion. I am now getting a new error:

make[4]: Leaving directory '/home/---/.julia/v0.6/MUMPS/src/MUMPS_5.0.1/src'
make[3]: Leaving directory '/home/---/.julia/v0.6/MUMPS/src/MUMPS_5.0.1/src'
make[2]: Leaving directory '/home/---/.julia/v0.6/MUMPS/src/MUMPS_5.0.1'
make[1]: Leaving directory '/home/---/.julia/v0.6/MUMPS/src/MUMPS_5.0.1'
gfortran -shared -fcray-pointer -O -fPIC  mumps_cmplx_p.f90  mumps_p.f90 call_mumps_cmplx_p.f90 call_mumps_p.f90   -o../lib/MUMPS -IMUMPS_5.0.1/include -LMUMPS_5.0.1/lib -ldmumps -lzmumps -lmumps_common -lpord  -L/usr/lib  -lmetis  -LMUMPS_5.0.1/libseq -lmpiseq  -lblas
/usr/bin/ld: MUMPS_5.0.1/lib/libpord.a(tree.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
MUMPS_5.0.1/lib/libpord.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
Makefile:14: recipe for target 'all' failed
make: *** [all] Error 1

Any clue about what's going on here? As you can see that gfortran call has the -fPIC flag added to it so I'm not sure what's going on. In my MUMPS Makefile.inc I have the lines

OPTF    = -O -fPIC -shared -DALLOW_NON_INIT
OPTL    = -O -fPIC
OPTC    = -O -fPIC

Is there somewhere else that I need to add -fPIC? Thanks!

[rShekhtman]

I have just downloaded the code, and was able to compile it. Why did you change the Makefiles? Since I have no idea what you did, my guess is that you added the -fPIC after you initially tried compiling the code. Can you go to the MUMPS_5.0.1/PORD/lib/ directory and remove the *.o files, since these files might not have been compiled with -fPIC. Now do make again.

[ianwilliamson]

I was originally tinkering with the Makefiles because I was having issues getting it to build. I've now started with a freshly checked out MUMPS.jl. It seems like the only line I need to change, out of the box, is in compiler_options.in:

LIBBLAS = /usr/lib/libblas/libblas.a

which is for the Ubuntu blas package. Doing this, and making sure that I clear the PORD objects after the run that produces the blas error addressed above, I still get the same error as before:

...

ranlib ../lib/libzmumps.a
make[4]: Leaving directory '/home/---/.julia/v0.6/MUMPS/src/MUMPS_5.0.1/src'
make[3]: Leaving directory '/home/---/.julia/v0.6/MUMPS/src/MUMPS_5.0.1/src'
make[2]: Leaving directory '/home/---/.julia/v0.6/MUMPS/src/MUMPS_5.0.1'make[1]: Leaving directory '/home/---/.julia/v0.6/MUMPS/src/MUMPS_5.0.1'
gfortran -O3 -fPIC -fdefault-integer-8 -shared -fcray-pointer  mumps_cmplx_p.f90  mumps_p.f90 call_mumps_cmplx_p.f90 call_mumps_p.f90   -o ../lib/MUMPS -IMUMPS_5.0.1/include -LMUMPS_5.0.1/lib -ldmumps -lzmumps -lmumps_common -lpord  -LMUMPS_5.0.1/../metis-5.1.0/build/Linux-x86_64/libmetis/ -lmetis  -LMUMPS_5.0.1/libseq -lmpiseq  /usr/lib/libblas/libblas.a
/usr/bin/ld: MUMPS_5.0.1/lib/libpord.a(tree.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a
shared object; recompile with -fPIC
MUMPS_5.0.1/lib/libpord.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
Makefile:14: recipe for target 'all' failed
make: *** [all] Error 1

[rShekhtman]

When you did make, and it was compiling the pord library, did the output have the -fPIC ? You should have seen lines such as:

gcc -I../include -O -I. -fPIC -DINTSIZE64  -c graph.c -o graph.o
...
gcc -I../include -O -I. -fPIC -DINTSIZE64  -c tree.c -o tree.o

[ianwilliamson]

Thank you again for your assistance with this so far! I do not see those particular gcc lines (graph.c and tree.c in particular), but on the gcc lines that I do have, I see the -fPIC option being applied. For reference, here is the output of running make: https://gist.githubusercontent.com/ianwilliamson/3300ea4e9eb521d6d74651d71ebcaf41/raw/7be9cadc0a7cf8962a4142f9b4dda7dfebb9c98c/make_log.txt

I'm really confused as to why we are observing different behavior.

[rShekhtman]

It doesn't look like it is compiling PORD. Go to the MUMPS_5.0.1/lib/ directory. Is there libpord.a ? If it is there, remove it. Again, make sure there are no *.o files in MUMPS_5.0.1/PORD/lib/ . Do make again. Do you see the above gcc lines?

[ianwilliamson]

Okay, so just to be extra sure I removed the MUMPS package and removed the MUMPS directory under .cache in julia and did a fresh clone of this git repository. Now it looks like it's compiling PORD but it's giving the -fPIC error on the libblas.a that I point it to:

...
a - zmumps_restart.o
ranlib ../lib/libzmumps.a
make[4]: Leaving directory '/home/---/.julia/v0.6/MUMPS/src/MUMPS_5.0.1/src'
make[3]: Leaving directory '/home/---/.julia/v0.6/MUMPS/src/MUMPS_5.0.1/src'
make[2]: Leaving directory '/home/---/.julia/v0.6/MUMPS/src/MUMPS_5.0.1'
make[1]: Leaving directory '/home/---/.julia/v0.6/MUMPS/src/MUMPS_5.0.1'
gfortran -O3 -fPIC -fdefault-integer-8 -shared -fcray-pointer  mumps_cmplx_p.f90  mumps_p.f90 call_mumps_cmplx_p.f90 call_mumps_p.f90   -o ../lib/MUMPS -IMUMPS_5.0.1/include -LMUMPS_5.0.1/lib -ldmumps -lzmumps -lmumps_common -lpord  -LMUMPS_5.0.1/../metis-5.1.0/build/Linux-x86_64/libmetis/ -lmetis  -LMUMPS_5.0.1/libseq -lmpiseq  /usr/lib/libblas/libblas.a 
/usr/bin/ld: /usr/lib/libblas/libblas.a(dgemm.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/lib/libblas/libblas.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
Makefile:14: recipe for target 'all' failed
make: *** [all] Error 1

Here's the full make log where I now see the gcc lines you mentioned above: https://gist.githubusercontent.com/ianwilliamson/fa602d0538bd4a08023cda991fb4ea58/raw/8a06d73103dc51b6e804916b1a2947e488feef95/make_log.txt

It turns out that for Ubuntu you need to specify -lblas rather than the path to /usr/lib/libblas/libblas.a. Therefore, compiler_options.in should have the line

LIBBLAS = -lblas

The package builds successfully and I was able to run Pkg.test("MUMPS") in Julia. Thanks again @rShekhtman for helping me with this!