amazon-archives / amazon-dsstne

Deep Scalable Sparse Tensor Network Engine (DSSTNE) is an Amazon developed library for building Deep Learning (DL) machine learning (ML) models
Apache License 2.0
4.41k stars 731 forks source link

make for train fails on ubuntu 16.04 #146

Open mkoo21 opened 6 years ago

mkoo21 commented 6 years ago

I believe I have followed the steps in the setup guide correctly, but I can't seem to get over the following issue at the final step:

mpiCC  -o train Utils.o NetCDFhelper.o NNRecsGenerator.o Filters.o Train.o ../lib/libdsstne.a   -L/usr/lib/atlas-base -L/usr/local/cuda/lib64 -L. -L/usr/local/lib/ -lcudnn -lcurand -lcublas -lcudart -lmpi -lmpi_cxx -ljsoncpp -lnetcdf_c++4 -lnetcdf -lblas -ldl -lstdc++  
../lib/libdsstne.a(NNNetwork.o): In function `LoadNeuralNetworkJSON(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, std::vector<NNDataSetBase*, std::allocator<NNDataSetBase*> > const&)':
NNNetwork.cpp:(.text+0xa103): undefined reference to `Json::ValueIteratorBase::name[abi:cxx11]() const'
NNNetwork.cpp:(.text+0xa3f9): undefined reference to `Json::ValueIteratorBase::name[abi:cxx11]() const'
NNNetwork.cpp:(.text+0xa669): undefined reference to `Json::ValueIteratorBase::name[abi:cxx11]() const'
NNNetwork.cpp:(.text+0xa7cf): undefined reference to `Json::ValueIteratorBase::name[abi:cxx11]() const'
NNNetwork.cpp:(.text+0xa8f8): undefined reference to `Json::ValueIteratorBase::name[abi:cxx11]() const'
../lib/libdsstne.a(NNNetwork.o):NNNetwork.cpp:(.text+0xaa11): more undefined references to `Json::ValueIteratorBase::name[abi:cxx11]() const' follow
collect2: error: ld returned 1 exit status
Makefile:30: recipe for target 'train' failed
make[1]: *** [train] Error 1

Any pointers on what's wrong?

scottlegrand commented 6 years ago

We just updated DSSTNE to build more easily on Ubuntu 16.04, but the current instructions are now out of date.

You just need to apt-get install libopenmpi-dev libnetcdf-dev libnetcdf-c++4-dev libjsoncpp-dev lipcppunit-dev

And it should just build cleanly.

crvaden commented 6 years ago

libcppunit-dev

quangnn1209 commented 6 years ago

Hi there, i also got error similar above

cd utils && make make[1]: Entering directory '/home/quang.nguyen/work/2018/dsstne/amazon-dsstne/src/amazon/dsstne/utils' **** RELEASE mode **** mkdir -p ../bin mpiCC -o train Utils.o NetCDFhelper.o NNRecsGenerator.o Filters.o Train.o ../lib/libdsstne.a -L/usr/lib/atlas-base -L/usr/local/cuda/lib64 -L. -L/usr/local/lib/ -lcudnn -lcurand -lcublas -lcudart -lmpi -lmpi_cxx -ljsoncpp -lnetcdf_c++4 -lnetcdf -lblas -ldl -lstdc++
/usr/bin/ld: warning: libnetcdf.so.11, needed by /usr/local/lib//libnetcdf_c++4.so, may conflict with libnetcdf.so.7 ../lib/libdsstne.a(NNNetwork.o): In function LoadNeuralNetworkJSON(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, std::vector<NNDataSetBase*, std::allocator<NNDataSetBase*> > const&)': NNNetwork.cpp:(.text+0xa103): undefined reference toJson::ValueIteratorBase::name[abi:cxx11]() const' NNNetwork.cpp:(.text+0xa3f9): undefined reference to Json::ValueIteratorBase::name[abi:cxx11]() const' NNNetwork.cpp:(.text+0xa669): undefined reference toJson::ValueIteratorBase::name[abi:cxx11]() const' NNNetwork.cpp:(.text+0xa7cf): undefined reference to Json::ValueIteratorBase::name[abi:cxx11]() const' NNNetwork.cpp:(.text+0xa8f8): undefined reference toJson::ValueIteratorBase::name[abi:cxx11]() const' ../lib/libdsstne.a(NNNetwork.o):NNNetwork.cpp:(.text+0xaa11): more undefined references to `Json::ValueIteratorBase::name[abi:cxx11]() const' follow collect2: error: ld returned 1 exit status Makefile:30: recipe for target 'train' failed make[1]: [train] Error 1 make[1]: Leaving directory '/home/quang.nguyen/work/2018/dsstne/amazon-dsstne/src/amazon/dsstne/utils' Makefile:17: recipe for target 'bin/train' failed make: [bin/train] Error 2

I tried solution scottlegrand mentioned but didn't solve the issue, can some one please give me an advice? Thanks a lot

scottlegrand commented 6 years ago

So I think you need to remove the use of /usr/local/lib. As of 16.04, you no longer need to download and install netcdd etc as they're part of the distribution.

On Jan 8, 2018 11:54 PM, "quangnn1209" notifications@github.com wrote:

Hi there, i also got error similar above cd utils && make make[1]: Entering directory '/home/quang.nguyen/work/2018/ dsstne/amazon-dsstne/src/amazon/dsstne/utils' **** RELEASE mode **** mkdir -p ../bin mpiCC -o train Utils.o NetCDFhelper.o NNRecsGenerator.o Filters.o Train.o ../lib/libdsstne.a -L/usr/lib/atlas-base -L/usr/local/cuda/lib64 -L. -L/usr/local/lib/ -lcudnn -lcurand -lcublas -lcudart -lmpi -lmpi_cxx -ljsoncpp -lnetcdf_c++4 -lnetcdf -lblas -ldl -lstdc++ /usr/bin/ld: warning: libnetcdf.so.11, needed by /usr/local/lib//libnetcdf_c++4.so, may conflict with libnetcdf.so.7 ../lib/libdsstne.a(NNNetwork.o): In functionLoadNeuralNetworkJSON( std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, unsigned int, std::vector<NNDataSetBase, std::allocator<NNDataSetBase> > const&)': NNNetwork.cpp:(.text+0xa103): undefined reference to Json::ValueIteratorBase::name[abi:cxx11]() const' NNNetwork.cpp:(.text+0xa3f9): undefined reference to Json::ValueIteratorBase::nameabi:cxx11 const' NNNetwork.cpp:(.text+0xa669): undefined reference to Json::ValueIteratorBase::name[abi:cxx11]() const' NNNetwork.cpp:(.text+0xa7cf): undefined reference to Json::ValueIteratorBase::nameabi:cxx11 const' NNNetwork.cpp:(.text+0xa8f8): undefined reference to Json::ValueIteratorBase::name[abi:cxx11]() const' ../lib/libdsstne.a(NNNetwork.o):NNNetwork.cpp:(.text+0xaa11): more undefined references toJson::ValueIteratorBase::nameabi:cxx11 const' follow

collect2: error: ld returned 1 exit status Makefile:30: recipe for target 'train' failed make[1]: [train] Error 1 make[1]: Leaving directory '/home/quang.nguyen/work/2018/ dsstne/amazon-dsstne/src/amazon/dsstne/utils' Makefile:17: recipe for target 'bin/train' failed make: [bin/train] Error 2 `

I tried solution scottlegrand mentioned but didn't solve the issue, can some one please give me an advice? Thanks a lot

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/amzn/amazon-dsstne/issues/146#issuecomment-356206714, or mute the thread https://github.com/notifications/unsubscribe-auth/ARNK9nqFcLSqfRJtjHKiUCnlygCBiu1aks5tIxsogaJpZM4Qat9K .

rgeorgej commented 6 years ago

Please ensure to remove the libjsoncpp from /usr/local/lib so that it gets linked with the installed one using libjsoncpp-dev.

quangnn1209 commented 6 years ago

Thank you scottlegrand, rgeorgej for your advices. It's working properly after "Remove libjsoncpp from /usr/local/lib"