search

openucx / ucx

Unified Communication X (mailing list - https://elist.ornl.gov/mailman/listinfo/ucx-group)
http://www.openucx.org
Other
1.15k stars 426 forks source link

UCX application link fails missing libbfd symbols #4244

Closed tjcw closed 5 years ago

tjcw commented 5 years ago

I am trying to build a modified IBM Spectrum MPI (based on OpenMPI) on top of UCX. The 'make conf' for the Spectrum MPI is failing when testing for the presence of one of the IBM libraries. On investigating, I find that the link for the ''conftest' is failing with indications that all the symbols from libbfd are missing in libucs.so . Here is the compile line and some of the error messages.

configure:300487: gcc -std=gnu99 -std=gnu99 -o conftest  -I/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/include/cuda  -I/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/include/cuda -I/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib/pami_port/include -I/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/include  -L/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib -L/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/mofed_400/usr/lib -L/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/mofed_400/opt/mellanox/mxm/lib -L/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/mofed_400/opt/mellanox/hcoll/lib -L/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/mofed_400/opt/mellanox/sharp/lib -L/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib/nvidia -L/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib   -L/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib  -L/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib -L/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib/pami_port/lib -L/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib conftest.c -lcollectives   -lucp -luct -lucm -lucs -lpami -lrt -ldl >&5
/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib/libucs.so: undefined reference to `_hex_value'
/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib/libucs.so: undefined reference to `unlink_if_ordinary'
/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib/libucs.so: undefined reference to `htab_traverse'
/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib/libucs.so: undefined reference to `make_relative_prefix'
/pmpi/scratch/cward/workspace/ibm-smpi-toucan-master/ompibase/dependencies/lib/libucs.so: undefined reference to `htab_create_alloc'

I had a further look at the UCX build, and it seems that the 'make install' for UCX relinks the UCX libraries (including libucs.so) without specifying '-lbfd'; thereby building a libucs.so which doesn't work for me. This is slightly complicated in that the build process for IBM Spectrum MPI builds its own 'binutils' (with libbfd) from source, so I'm not using the 'system' libbfd. My build works on IBM POWER; I am only having the problem on x86. I am using the 'master' branch of UCX at commit 9449996970a260e2688895104424335c3aa04cf0 . What do I need to change, to get my 'make conf' to work ?

tjcw commented 5 years ago

The application build is working now; I'm not sure what has changed, the only thing I know I have done is to rebase to a newer openucx 'master' branch level. Closing.