How to use milvus from another directory

[abuelnasr0]

I installed milvus successfully and ran the simple example. but I want to use milvus from another directory. I tried this to build the program:

g++  main.cpp -I /usr/local/include  -L /usr/local/lib -l milvus_sdk -o output 

but I got nearly 10000 lines of undefined reference error because of the SDK dependencies. how to show the SDK dependencies that are in the cmake_build directory to the linker?

[matrixji]

This is a linker/loader question. For gcc toolchain, you could add RPATH by linker with -Wl,-rpath=/usr/local/lib for your case, e.g.: g++ main.cpp -I /usr/local/include -L /usr/local/lib -l milvus_sdk -o output -Wl,-rpath=/usr/local/lib

Of course, you could also try to use the LD_LIBRARY_PATH environment variable while running the application.

[abuelnasr0]

@matrixji Thanks for the reply. This solution doesn't solve the problem because all the dependencies are in the cmake_build directory. none of them are in the usr/local/lib directory, although when I open any .pc file in cmake_build directory it says that the libraries are in the /user/local/lib directory.

is that expected of build? or should all the dependencies be in the usr/local/lib directory?

[abuelnasr0]

I used this bash script to include all necessary libs and include files from the cmake_dir.

#!/bin/bash
CMAK_BIULD_DIR=/home/mohamed/Mohamed/HPCC/milvus-sdk-cpp/cmake_build

# Libs directories.
LIBS_DIR=$(find ${CMAK_BIULD_DIR} -type f -name '*.a' | sed -r 's|/[^/]+$||' |sort |uniq)
BUILD_LIBS_DIR=""

for DIR in $LIBS_DIR;
do BUILD_LIBS_DIR="$BUILD_LIBS_DIR -L$DIR"
done

# Libs names.
LIBS_NAMES=$(find  ${CMAK_BIULD_DIR} -type f -name '*.a' -printf "%f\n")
BUILD_LIBS_NAMES=""
for NAME in $LIBS_NAMES;
do 
TMP_NAME=${NAME#"lib"}
firstchar=${TMP_NAME:0:1}
if [[ $firstchar == "0" ]];then
    continue
else
    BUILD_LIBS_NAMES="$BUILD_LIBS_NAMES -l${TMP_NAME%".a"}"
fi
done

#Include direcotries.
INCLUDE_DIRS=$(find ${CMAK_BIULD_DIR} -type f -name '*.h' | grep include | sed 's/include.*/include/' |sort |uniq  )
INCLUDE_FILES_DIR=""

for DIR in $INCLUDE_DIRS;
do INCLUDE_FILES_DIR="$INCLUDE_FILES_DIR -I$DIR"
done

#build the code.
g++  main.cpp ${INCLUDE_FILES_DIR} ${BUILD_LIBS_DIR} ${BUILD_LIBS_NAMES} -o output &> error.txt

But it will be better to install the dependencies to usr/local during build, won't it? I also tried adding -DBUILD_SHARED_LIBS=YES during cmake from milvus-io/milvus-sdk-cpp#241 and it didn't work either.

[LuongDucLong]

Have same problem, I have both /usr/local/lib/libmilvus_sdk.a and /usr/local/include/milvus but cannot compile main.cpp, got nearly 10000 lines of undefined reference error after run: g++ -std=c++14 main.cpp -I /usr/local/include -L /usr/local/lib -l milvus_sdk -o output

[LuongDucLong]

Nobody cares about this project?

[xiaofan-luan]

did you compile successfully under your environment? https://github.com/milvus-io/milvus-sdk-cpp/blob/master/DEVELOPMENT.md

[LuongDucLong]

did you compile successfully under your environment? https://github.com/milvus-io/milvus-sdk-cpp/blob/master/DEVELOPMENT.md

Yes. I followed the installation in DEVELOPMENT.md. This is what I run:

git clone https://github.com/milvus-io/milvus-sdk-cpp
cd milvus-sdk-cpp
git checkout main
make all-release

I checked I had both /usr/local/lib/libmilvus_sdk.a and /usr/local/include/milvus. And I also can run ./sdk_simple. But when I run: g++ main.cpp -I /usr/local/include -L /usr/local/lib -l milvus_sdk -o output -Wl,-rpath=/usr/local/lib or run the @abuelnasr0 's bash script above, still got the errors with 13000 lines.

Tell me if I do something wrong? Thank you

[xiaofan-luan]

what operating system and gcc versions you are using?

could you share the error message?

[xiaofan-luan]

and is there a reason you specify to use -std=c++14?

[LuongDucLong]

My OS is Ubuntu 20.04 and gcc version is 9.4.0 I tried both -std and not. but all return errors. This is the head and the end of the errors:

/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(MilvusClientImpl.cpp.o): in function `milvus::MilvusClientImpl::CalcDistance(milvus::CalcDistanceArguments const&, milvus::DistanceArray&)::{lambda()#2}::operator()() const::{lambda(milvus::proto::milvus::VectorsArray*, std::shared_ptr<milvus::Field>&&, bool)#1}::operator()(milvus::proto::milvus::VectorsArray, std::shared_ptr<milvus::Field>, bool) const':
MilvusClientImpl.cpp:(.text+0x3a61): undefined reference to `google::protobuf::internal::ArenaStringPtr::Mutable[abi:cxx11](google::protobuf::Arena*)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x3c10): undefined reference to `google::protobuf::RepeatedField<float>::Reserve(int)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x3cd3): undefined reference to `google::protobuf::RepeatedField<float>::Reserve(int)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x3d1b): undefined reference to `google::protobuf::internal::fixed_address_empty_string[abi:cxx11]'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x3e5b): undefined reference to `google::protobuf::internal::ArenaStringPtr::Set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Arena*)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x3e84): undefined reference to `google::protobuf::RepeatedPtrField<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::Add()'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x4000): undefined reference to `google::protobuf::internal::ArenaStringPtr::Set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Arena*)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x404d): undefined reference to `google::protobuf::RepeatedField<long>::Reserve(int)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x4185): undefined reference to `google::protobuf::RepeatedField<long>::Reserve(int)'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(MilvusClientImpl.cpp.o): in function `std::_Function_handler<void (milvus::proto::milvus::ShowCollectionsResponse const&), milvus::MilvusClientImpl::ShowCollections(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<milvus::CollectionInfo, std::allocator<milvus::CollectionInfo> >&)::{lambda(milvus::proto::milvus::ShowCollectionsResponse const&)#2}>::_M_invoke(std::_Any_data const&, milvus::proto::milvus::ShowCollectionsResponse const&)':
MilvusClientImpl.cpp:(.text+0x4394): undefined reference to `google::protobuf::RepeatedPtrField<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::Get(int) const'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(MilvusClientImpl.cpp.o): in function `std::_Function_handler<void (milvus::proto::milvus::ShowPartitionsResponse const&), milvus::MilvusClientImpl::ShowPartitions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::vector<milvus::PartitionInfo, std::allocator<milvus::PartitionInfo> >&)::{lambda(milvus::proto::milvus::ShowPartitionsResponse const&)#2}>::_M_invoke(std::_Any_data const&, milvus::proto::milvus::ShowPartitionsResponse const&)':
MilvusClientImpl.cpp:(.text+0x44d7): undefined reference to `google::protobuf::RepeatedPtrField<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::size() const'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x4638): undefined reference to `google::protobuf::RepeatedPtrField<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::Get(int) const'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(MilvusClientImpl.cpp.o): in function `std::_Function_handler<void (milvus::proto::milvus::ListCredUsersResponse const&), milvus::MilvusClientImpl::ListCredUsers(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&)::{lambda(milvus::proto::milvus::ListCredUsersResponse const&)#2}>::_M_invoke(std::_Any_data const&, milvus::proto::milvus::ListCredUsersResponse const&)':
MilvusClientImpl.cpp:(.text+0x592f): undefined reference to `google::protobuf::RepeatedPtrField<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::begin() const'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x593a): undefined reference to `google::protobuf::RepeatedPtrField<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::end() const'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(MilvusClientImpl.cpp.o): in function `milvus::MilvusClientImpl::Flush(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, milvus::ProgressMonitor const&)::{lambda(milvus::proto::milvus::FlushResponse const&)#2}::operator()(milvus::proto::milvus::FlushResponse const&) const [clone .isra.0]':
MilvusClientImpl.cpp:(.text+0x5ae0): undefined reference to `google::protobuf::internal::MapFieldBase::SyncMapWithRepeatedField() const'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(MilvusClientImpl.cpp.o): in function `milvus::MilvusClientImpl::Search(milvus::SearchArguments const&, milvus::SearchResults&, int)::{lambda(milvus::proto::milvus::SearchResults const&)#3}::operator()(milvus::proto::milvus::SearchResults const&) const [clone .isra.0]':
MilvusClientImpl.cpp:(.text+0x72ad): undefined reference to `google::protobuf::internal::LogMessage::LogMessage(google::protobuf::LogLevel, char const*, int)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x72bc): undefined reference to `google::protobuf::internal::LogMessage::operator<<(char const*)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x72c7): undefined reference to `google::protobuf::internal::LogFinisher::operator=(google::protobuf::internal::LogMessage&)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x72cf): undefined reference to `google::protobuf::internal::LogMessage::~LogMessage()'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x72f5): undefined reference to `google::protobuf::internal::LogMessage::LogMessage(google::protobuf::LogLevel, char const*, int)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x7304): undefined reference to `google::protobuf::internal::LogMessage::operator<<(char const*)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x730f): undefined reference to `google::protobuf::internal::LogFinisher::operator=(google::protobuf::internal::LogMessage&)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x7317): undefined reference to `google::protobuf::internal::LogMessage::~LogMessage()'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(MilvusClientImpl.cpp.o): in function `std::_Function_handler<void (milvus::proto::milvus::DescribeCollectionResponse const&), milvus::MilvusClientImpl::DescribeCollection(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, milvus::CollectionDesc&)::{lambda(milvus::proto::milvus::DescribeCollectionResponse const&)#2}>::_M_invoke(std::_Any_data const&, milvus::proto::milvus::DescribeCollectionResponse const&)':
MilvusClientImpl.cpp:(.text+0x7631): undefined reference to `google::protobuf::RepeatedPtrField<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::size() const'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x766d): undefined reference to `google::protobuf::RepeatedPtrField<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::end() const'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x767d): undefined reference to `google::protobuf::RepeatedPtrField<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::begin() const'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(MilvusClientImpl.cpp.o): in function `milvus::MilvusClientImpl::GetVersion(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)':
MilvusClientImpl.cpp:(.text+0x7c8d): undefined reference to `google::protobuf::internal::ZeroFieldsBase::~ZeroFieldsBase()'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(MilvusClientImpl.cpp.o): in function `std::_Function_handler<milvus::proto::milvus::CreateCollectionRequest (), milvus::MilvusClientImpl::CreateCollection(milvus::CollectionSchema const&)::{lambda()#1}>::_M_invoke(std::_Any_data const&)':
MilvusClientImpl.cpp:(.text+0x7fcc): undefined reference to `google::protobuf::internal::ArenaStringPtr::Set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Arena*)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x8013): undefined reference to `google::protobuf::internal::ArenaStringPtr::Set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Arena*)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x803c): undefined reference to `google::protobuf::internal::ArenaStringPtr::Set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Arena*)'
/usr/bin/ld: MilvusClientImpl.cpp:(.text+0x80ba): undefined reference to `google::protobuf::internal::ArenaStringPtr::Set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Arena*)'
...
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(feder.pb.cc.o):(.data.rel.ro._ZTVN6milvus5proto5feder32DescribeSegmentIndexDataResponseE[_ZTVN6milvus5proto5feder32DescribeSegmentIndexDataResponseE]+0x40): undefined reference to `google::protobuf::Message::InitializationErrorString[abi:cxx11]() const'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(feder.pb.cc.o):(.data.rel.ro._ZTVN6milvus5proto5feder32DescribeSegmentIndexDataResponseE[_ZTVN6milvus5proto5feder32DescribeSegmentIndexDataResponseE]+0x48): undefined reference to `google::protobuf::Message::CheckTypeAndMergeFrom(google::protobuf::MessageLite const&)'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(feder.pb.cc.o):(.data.rel.ro._ZTVN6milvus5proto5feder32DescribeSegmentIndexDataResponseE[_ZTVN6milvus5proto5feder32DescribeSegmentIndexDataResponseE]+0x78): undefined reference to `google::protobuf::Message::MergeFrom(google::protobuf::Message const&)'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(feder.pb.cc.o):(.data.rel.ro._ZTVN6milvus5proto5feder32DescribeSegmentIndexDataResponseE[_ZTVN6milvus5proto5feder32DescribeSegmentIndexDataResponseE]+0x80): undefined reference to `google::protobuf::Message::SpaceUsedLong() const'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(feder.pb.cc.o):(.data.rel.ro+0x0): undefined reference to `google::protobuf::Message::CopyWithSourceCheck(google::protobuf::Message&, google::protobuf::Message const&)'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(feder.pb.cc.o):(.data.rel.ro+0x10): undefined reference to `google::protobuf::Message::CopyWithSourceCheck(google::protobuf::Message&, google::protobuf::Message const&)'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(feder.pb.cc.o):(.data.rel.ro+0x20): undefined reference to `google::protobuf::Message::CopyWithSourceCheck(google::protobuf::Message&, google::protobuf::Message const&)'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(feder.pb.cc.o):(.data.rel.ro+0x30): undefined reference to `google::protobuf::Message::CopyWithSourceCheck(google::protobuf::Message&, google::protobuf::Message const&)'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(feder.pb.cc.o):(.data.rel.ro+0x40): undefined reference to `google::protobuf::Message::CopyWithSourceCheck(google::protobuf::Message&, google::protobuf::Message const&)'
/usr/bin/ld: /home/longnp2/longld8/milvus-sdk-cpp/cmake_build/src/libmilvus_sdk.a(feder.pb.cc.o):(.data.rel.ro+0x50): more undefined references to `google::protobuf::Message::CopyWithSourceCheck(google::protobuf::Message&, google::protobuf::Message const&)' follow
collect2: error: ld returned 1 exit status

[matrixji]

IMO, this is a code integration issue, we do tests/examples with CMake, and this is smooth for application integration with milvus-sdk-cpp, but if guys use GNU Makefile to manually compile/link commands, the dependency may lost. I'll try to provide some guide documentation based on Ubuntu. Maybe a couple of weeks later...