search

psathyrella / partis

B- and T-cell receptor sequence annotation, simulation, clonal family and germline inference, and affinity prediction
GNU General Public License v3.0
54 stars 34 forks source link

Building bcrham failing with "undefined reference to YAML" #277

Closed scharch closed 5 years ago

scharch commented 5 years ago

I just pulled the latest commit, but I can't build bcrham:

$ bin/build.sh 

--> running bin/build.sh

--> building ig-sw
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
scons: done building targets.

--> building ham
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
g++ -o bcrham -Ofast -std=c++11 _build/bcrham.o -L_build -lham -lyaml-cpp -lgsl -lgslcblas
_build/libham.a(model.o): In function `YAML::Node YAML::Node::operator[]<char [15]>(char const (&) [15]) [clone .constprop.225]':
model.cc:(.text+0x122e): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(model.o): In function `YAML::TypedBadConversion<double>::~TypedBadConversion()':
model.cc:(.text._ZN4YAML18TypedBadConversionIdED2Ev[_ZN4YAML18TypedBadConversionIdED5Ev]+0x8): undefined reference to `YAML::BadConversion::~BadConversion()'
_build/libham.a(model.o): In function `YAML::TypedBadConversion<double>::~TypedBadConversion()':
model.cc:(.text._ZN4YAML18TypedBadConversionIdED0Ev[_ZN4YAML18TypedBadConversionIdED5Ev]+0xc): undefined reference to `YAML::BadConversion::~BadConversion()'
_build/libham.a(model.o): In function `YAML::TypedBadConversion<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::~TypedBadConversion()':
model.cc:(.text._ZN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEED2Ev[_ZN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEED5Ev]+0x8): undefined reference to `YAML::BadConversion::~BadConversion()'
_build/libham.a(model.o): In function `YAML::TypedBadConversion<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::~TypedBadConversion()':
model.cc:(.text._ZN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEED0Ev[_ZN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEED5Ev]+0xc): undefined reference to `YAML::BadConversion::~BadConversion()'
_build/libham.a(model.o): In function `YAML::BadConversion::BadConversion(YAML::Mark const&)':
model.cc:(.text._ZN4YAML13BadConversionC2ERKNS_4MarkE[_ZN4YAML13BadConversionC5ERKNS_4MarkE]+0x119): undefined reference to `vtable for YAML::BadConversion'
_build/libham.a(model.o): In function `YAML::Node YAML::Node::operator[]<char [15]>(char const (&) [15])':
model.cc:(.text._ZN4YAML4NodeixIA15_cEES0_RKT_[_ZN4YAML4NodeixIA15_cEES0_RKT_]+0x611): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(model.o): In function `YAML::detail::node& YAML::detail::node_data::get<unsigned long>(unsigned long const&, std::shared_ptr<YAML::detail::memory_holder>)':
model.cc:(.text._ZN4YAML6detail9node_data3getImEERNS0_4nodeERKT_St10shared_ptrINS0_13memory_holderEE[_ZN4YAML6detail9node_data3getImEERNS0_4nodeERKT_St10shared_ptrINS0_13memory_holderEE]+0xe0e): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(model.o):(.rodata._ZTIN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEE[_ZTIN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEE]+0x10): undefined reference to `typeinfo for YAML::BadConversion'
_build/libham.a(model.o):(.rodata._ZTIN4YAML18TypedBadConversionIdEE[_ZTIN4YAML18TypedBadConversionIdEE]+0x10): undefined reference to `typeinfo for YAML::BadConversion'
_build/libham.a(state.o): In function `ham::State::Parse(YAML::Node, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, ham::Track*)':
state.cc:(.text+0x3bb0): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(state.o): In function `YAML::detail::node& YAML::detail::node_data::get<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<YAML::detail::memory_holder>)':
state.cc:(.text._ZN4YAML6detail9node_data3getINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEERNS0_4nodeERKT_St10shared_ptrINS0_13memory_holderEE[_ZN4YAML6detail9node_data3getINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEERNS0_4nodeERKT_St10shared_ptrINS0_13memory_holderEE]+0x3ab): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(emission.o): In function `ham::Emission::Parse(YAML::Node, ham::Track*)':
emission.cc:(.text+0xd45): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
collect2: error: ld returned 1 exit status
scons: *** [bcrham] Error 1
scons: building terminated because of errors.

I even created a fresh conda environment directly from the Dockerfile, but got the same error. libyaml-cpp-dev, libyaml-dev, and pyyaml are all installed properly, as far as I can tell.

psathyrella commented 5 years ago

This is probably related to the switch (I think last year some time?) from a local copy (in ham) of yaml-cpp to a package-manager installed version. My first guess is that the ham build directory, which is in the linker's path, has the old locally-compiled version of yaml-cpp, so it's mistakenly trying to link against that. So rm'ing the _build directory in packages/ham and recompiling should fix it (or probably just rm the yaml-cpp related objects/libs in there and re linking). Let me know if that doesn't work though.

scharch commented 5 years ago

No joy :( Is it possible my locally-compiled version is conflicting with the apt-installed one somehow?

psathyrella commented 5 years ago

Yeah, very possible. The functions that the linker is not finding could very well be only present in the newer version. Presumably you have two versions -- one in your conda env, and one in the system lib directories -- and you should be able to switch which is used by switching the order that the linker searches for them in. The easiest way I know to do that is with LD_LIBRARY_PATH. I don't know how to have the linker print which version it's finding from which path, but that is presumably also possible. So I'd do a find -name libyaml-cpp* <conda dir, system lib dir> and try both of those at the start of LD_LIBRARY_PATH. It may also be possible to update both/one of the conda/system versions to a more recent version so they match.

scharch commented 5 years ago

It's referencing the right version of yaml-cpp, but I'm still getting the same error message:

schrammca@cogsworth:/home/schrammca/partis/partis/packages/ham$ g++ -o bcrham -Ofast -std=c++11 _build/bcrham.o -L_build -lham -lyaml-cpp -lgsl -lgslcblas -Wl,--verbose | grep libyaml
attempt to open _build/libyaml-cpp.so failed
attempt to open _build/libyaml-cpp.a failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/5/libyaml-cpp.so failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/5/libyaml-cpp.a failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libyaml-cpp.so succeeded
-lyaml-cpp (/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libyaml-cpp.so)
_build/libham.a(model.o): In function `YAML::Node YAML::Node::operator[]<char [15]>(char const (&) [15]) [clone .constprop.225]':
model.cc:(.text+0x122e): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(model.o): In function `YAML::TypedBadConversion<double>::~TypedBadConversion()':
model.cc:(.text._ZN4YAML18TypedBadConversionIdED2Ev[_ZN4YAML18TypedBadConversionIdED5Ev]+0x8): undefined reference to `YAML::BadConversion::~BadConversion()'
_build/libham.a(model.o): In function `YAML::TypedBadConversion<double>::~TypedBadConversion()':
model.cc:(.text._ZN4YAML18TypedBadConversionIdED0Ev[_ZN4YAML18TypedBadConversionIdED5Ev]+0xc): undefined reference to `YAML::BadConversion::~BadConversion()'
_build/libham.a(model.o): In function `YAML::TypedBadConversion<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::~TypedBadConversion()':
model.cc:(.text._ZN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEED2Ev[_ZN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEED5Ev]+0x8): undefined reference to `YAML::BadConversion::~BadConversion()'
_build/libham.a(model.o): In function `YAML::TypedBadConversion<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::~TypedBadConversion()':
model.cc:(.text._ZN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEED0Ev[_ZN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEED5Ev]+0xc): undefined reference to `YAML::BadConversion::~BadConversion()'
_build/libham.a(model.o): In function `YAML::BadConversion::BadConversion(YAML::Mark const&)':
model.cc:(.text._ZN4YAML13BadConversionC2ERKNS_4MarkE[_ZN4YAML13BadConversionC5ERKNS_4MarkE]+0x119): undefined reference to `vtable for YAML::BadConversion'
_build/libham.a(model.o): In function `YAML::Node YAML::Node::operator[]<char [15]>(char const (&) [15])':
model.cc:(.text._ZN4YAML4NodeixIA15_cEES0_RKT_[_ZN4YAML4NodeixIA15_cEES0_RKT_]+0x611): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(model.o): In function `YAML::detail::node& YAML::detail::node_data::get<unsigned long>(unsigned long const&, std::shared_ptr<YAML::detail::memory_holder>)':
model.cc:(.text._ZN4YAML6detail9node_data3getImEERNS0_4nodeERKT_St10shared_ptrINS0_13memory_holderEE[_ZN4YAML6detail9node_data3getImEERNS0_4nodeERKT_St10shared_ptrINS0_13memory_holderEE]+0xe0e): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(model.o):(.rodata._ZTIN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEE[_ZTIN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEE]+0x10): undefined reference to `typeinfo for YAML::BadConversion'
_build/libham.a(model.o):(.rodata._ZTIN4YAML18TypedBadConversionIdEE[_ZTIN4YAML18TypedBadConversionIdEE]+0x10): undefined reference to `typeinfo for YAML::BadConversion'
_build/libham.a(state.o): In function `ham::State::Parse(YAML::Node, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, ham::Track*)':
state.cc:(.text+0x3bbf): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(state.o): In function `YAML::detail::node& YAML::detail::node_data::get<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<YAML::detail::memory_holder>)':
state.cc:(.text._ZN4YAML6detail9node_data3getINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEERNS0_4nodeERKT_St10shared_ptrINS0_13memory_holderEE[_ZN4YAML6detail9node_data3getINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEERNS0_4nodeERKT_St10shared_ptrINS0_13memory_holderEE]+0x36e): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(emission.o): In function `ham::Emission::Parse(YAML::Node, ham::Track*)':
emission.cc:(.text+0xda8): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
collect2: error: ld returned 1 exit status
psathyrella commented 5 years ago

arg :-(

just to be sure, the stuff this particular g++ call is attempting to link using /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libyaml-cpp.so was definitely also compiled with the version at that location?

psathyrella commented 5 years ago

The main thing it seems not to find is convert_to_map(), which isn't something I'm actually calling directly, so it's got to be a version thing I think? Trying to see what version that was new in.

This sounds pretty similar, although I'm not sure it's super actionable.

scharch commented 5 years ago

just to be sure, the stuff this particular g++ call is attempting to link using /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libyaml-cpp.so was definitely also compiled with the version at that location?

Not sure I understand, but: that copy of libyaml-cpp.so is the one installed by apt-get and is the only one I can find on my computer. I also deleted the directory packages/ham/_build and then re-ran bin/build.sh, with the same results.

scharch commented 5 years ago

Oh sheesh I figured it out: the libyaml-cpp-dev distro available for xenial is 0.5.2. I compiled and installed 0.6.2 locally and now we're back in business. Maybe you can include an explicit version check?

psathyrella commented 5 years ago

huh, that's super weird, sorry about that. On my desktop (also xenial), as far as i can tell 0.5.2 is working fine. Here's the partial output of that (amazing) g++ option:

attempt to open _build/libyaml-cpp.so failed
attempt to open _build/libyaml-cpp.a failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/5/libyaml-cpp.so failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/5/libyaml-cpp.a failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libyaml-cpp.so succeeded
-lyaml-cpp (/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libyaml-cpp.so)

and if I ls the dir file it seems to be using:

lrwxrwxrwx 1 root root 18 Dec  1  2015 /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libyaml-cpp.so -> libyaml-cpp.so.0.5
lrwxrwxrwx 1 root root 20 Dec  1  2015 /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libyaml-cpp.so.0.5 -> libyaml-cpp.so.0.5.2

The Dockerfile (which is what gets run by dockerhub on every push to master) is pulling from continuumio/anaconda which is using debian:latest, which seems to also be using 0.5.2.

And even on our server, where I don't have sudo access so for some reason I'm using a horrible hack where I copied the one .so file to a special dir and add it by hand to the scons file in packages/ham/, it's also 0.5.

The existence of 0.6 was the reason I moved off the local compiled version of yaml-cpp, since 0.6 removes the boost dependency, but then I had to add boost anyway, if I'm remembering right, since 0.6 wasn't yet in APT.

But I'm at a loss why it's not linking correctly for you. As far as I can tell the offending fcn has been there for five years?

psathyrella commented 5 years ago

The user in #282 seems to also be on 0.5, so whatever the issue was seems like was at least not universal.

YaroslavHladun commented 3 years ago

I just pulled the latest commit, but I can't build bcrham:

$ bin/build.sh 

--> running bin/build.sh

--> building ig-sw
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
scons: done building targets.

--> building ham
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
g++ -o bcrham -Ofast -std=c++11 _build/bcrham.o -L_build -lham -lyaml-cpp -lgsl -lgslcblas
_build/libham.a(model.o): In function `YAML::Node YAML::Node::operator[]<char [15]>(char const (&) [15]) [clone .constprop.225]':
model.cc:(.text+0x122e): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(model.o): In function `YAML::TypedBadConversion<double>::~TypedBadConversion()':
model.cc:(.text._ZN4YAML18TypedBadConversionIdED2Ev[_ZN4YAML18TypedBadConversionIdED5Ev]+0x8): undefined reference to `YAML::BadConversion::~BadConversion()'
_build/libham.a(model.o): In function `YAML::TypedBadConversion<double>::~TypedBadConversion()':
model.cc:(.text._ZN4YAML18TypedBadConversionIdED0Ev[_ZN4YAML18TypedBadConversionIdED5Ev]+0xc): undefined reference to `YAML::BadConversion::~BadConversion()'
_build/libham.a(model.o): In function `YAML::TypedBadConversion<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::~TypedBadConversion()':
model.cc:(.text._ZN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEED2Ev[_ZN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEED5Ev]+0x8): undefined reference to `YAML::BadConversion::~BadConversion()'
_build/libham.a(model.o): In function `YAML::TypedBadConversion<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::~TypedBadConversion()':
model.cc:(.text._ZN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEED0Ev[_ZN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEED5Ev]+0xc): undefined reference to `YAML::BadConversion::~BadConversion()'
_build/libham.a(model.o): In function `YAML::BadConversion::BadConversion(YAML::Mark const&)':
model.cc:(.text._ZN4YAML13BadConversionC2ERKNS_4MarkE[_ZN4YAML13BadConversionC5ERKNS_4MarkE]+0x119): undefined reference to `vtable for YAML::BadConversion'
_build/libham.a(model.o): In function `YAML::Node YAML::Node::operator[]<char [15]>(char const (&) [15])':
model.cc:(.text._ZN4YAML4NodeixIA15_cEES0_RKT_[_ZN4YAML4NodeixIA15_cEES0_RKT_]+0x611): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(model.o): In function `YAML::detail::node& YAML::detail::node_data::get<unsigned long>(unsigned long const&, std::shared_ptr<YAML::detail::memory_holder>)':
model.cc:(.text._ZN4YAML6detail9node_data3getImEERNS0_4nodeERKT_St10shared_ptrINS0_13memory_holderEE[_ZN4YAML6detail9node_data3getImEERNS0_4nodeERKT_St10shared_ptrINS0_13memory_holderEE]+0xe0e): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(model.o):(.rodata._ZTIN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEE[_ZTIN4YAML18TypedBadConversionINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEE]+0x10): undefined reference to `typeinfo for YAML::BadConversion'
_build/libham.a(model.o):(.rodata._ZTIN4YAML18TypedBadConversionIdEE[_ZTIN4YAML18TypedBadConversionIdEE]+0x10): undefined reference to `typeinfo for YAML::BadConversion'
_build/libham.a(state.o): In function `ham::State::Parse(YAML::Node, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, ham::Track*)':
state.cc:(.text+0x3bb0): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(state.o): In function `YAML::detail::node& YAML::detail::node_data::get<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<YAML::detail::memory_holder>)':
state.cc:(.text._ZN4YAML6detail9node_data3getINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEERNS0_4nodeERKT_St10shared_ptrINS0_13memory_holderEE[_ZN4YAML6detail9node_data3getINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEERNS0_4nodeERKT_St10shared_ptrINS0_13memory_holderEE]+0x3ab): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
_build/libham.a(emission.o): In function `ham::Emission::Parse(YAML::Node, ham::Track*)':
emission.cc:(.text+0xd45): undefined reference to `YAML::detail::node_data::convert_to_map(std::shared_ptr<YAML::detail::memory_holder>)'
collect2: error: ld returned 1 exit status
scons: *** [bcrham] Error 1
scons: building terminated because of errors.

I even created a fresh conda environment directly from the Dockerfile, but got the same error. libyaml-cpp-dev, libyaml-dev, and pyyaml are all installed properly, as far as I can tell.

Hi! Are there any updates about this? I have the same problem.

psathyrella commented 3 years ago

I don't have any updates, no. Near as I could tell though it still stemmed from somehow having two versions installed on the system, or compiling and running on different systems (as here), or something similar. It could for sure be a problem on my end, or at least something that I could add a check for, but at least as last I looked I couldn't figure out anything I could do. Let me know if there's anything I can do to help though.