yahoo / CaffeOnSpark

Distributed deep learning on Hadoop and Spark clusters.
Apache License 2.0
1.27k stars 358 forks source link

Could anyone help about build CaffeOnSpak while caffe-distri failed ? #288

Open kioipp opened 6 years ago

kioipp commented 6 years ago

Hi , I got a problem similar with others while building , but tried many ways still can't work , could anyone help me , it would be great helpful .

.build_release/lib/libcaffe.so: undefined reference to cv::imread(cv::String const&, int)' .build_release/lib/libcaffe.so: undefined reference tocv::String::allocate(unsigned long)' .build_release/lib/libcaffe.so: undefined reference to cv::String::deallocate()' .build_release/lib/libcaffe.so: undefined reference tocv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator >&, std::vector<int, std::allocator > const&)' collect2: error: ld returned 1 exit status make[1]: [.build_release/tools/caffe.bin] Error 1 make[1]: Waiting for unfinished jobs.... .build_release/lib/libcaffe.so: undefined reference to cv::imread(cv::String const&, int)' .build_release/lib/libcaffe.so: undefined reference tocv::String::allocate(unsigned long)' .build_release/lib/libcaffe.so: undefined reference to cv::String::deallocate()' .build_release/lib/libcaffe.so: undefined reference tocv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator >&, std::vector<int, std::allocator > const&)' collect2: error: ld returned 1 exit status make[1]: [.build_release/tools/compute_image_mean.bin] Error 1 .build_release/lib/libcaffe.so: undefined reference to cv::imread(cv::String const&, int)' .build_release/lib/libcaffe.so: undefined reference tocv::String::allocate(unsigned long)' .build_release/lib/libcaffe.so: undefined reference to cv::String::deallocate()' .build_release/lib/libcaffe.so: undefined reference tocv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator >&, std::vector<int, std::allocator > const&)' collect2: error: ld returned 1 exit status make[1]: [.build_release/tools/convert_imageset.bin] Error 1 make[1]: Leaving directory `/home/hadoop/CaffeOnSpark/caffe-public' export LD_LIBRARY_PATH=":/usr/local/lib:/home/hadoop/CaffeOnSpark/caffe-public/distribute/lib:/home/hadoop/CaffeOnSpark/caffe-distri/distribute/lib:/usr/lib64:/lib64 "; GLOG_minloglevel=1 mvn -B package -DskipTests [INFO] Scanning for projects... [WARNING] [WARNING] Some problems were encountered while building the effective model for com.yahoo.ml:caffe-grid:jar:0.1-SNAPSHOT [WARNING] The expression ${version} is deprecated. Please use ${project.version} instead. [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. [WARNING] [INFO] ------------------------------------------------------------------------ [INFO] Reactor Build Order: [INFO] [INFO] caffe [INFO] caffe-distri [INFO] caffe-grid [INFO]
[INFO] ------------------------------------------------------------------------ [INFO] Building caffe 0.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO]
[INFO] ------------------------------------------------------------------------ [INFO] Building caffe-distri 0.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-antrun-plugin:1.7:run (proto) @ caffe-distri --- [INFO] Executing tasks

protoc: [exec] make[1]: Entering directory /home/hadoop/CaffeOnSpark/caffe-distri' [exec] make[1]: Leaving directory/home/hadoop/CaffeOnSpark/caffe-distri' [exec] make[1]: No rule to make target ../caffe-public/distribute/proto/caffe.proto', needed bysrc/main/java/caffe/Caffe.java'. Stop. [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] caffe .............................................. SUCCESS [ 0.001 s] [INFO] caffe-distri ....................................... FAILURE [ 1.617 s] [INFO] caffe-grid ......................................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.123 s [INFO] Finished at: 2017-11-15T09:42:48+00:00 [INFO] Final Memory: 8M/54M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (proto) on project caffe-distri: An Ant BuildException has occured: exec returned: 2 [ERROR] around Ant part ...... @ 5:91 in /home/hadoop/CaffeOnSpark/caffe-distri/target/antrun/build-protoc.xml [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :caffe-distri make: [build] Error 1

junshi15 commented 6 years ago

check your opencv version. I think the current CaffeOnSpark only supports openCV 2.5. You may have a newer version.

JJJzheng commented 6 years ago

chech your protobuf: protoc –version make sure protocbuf 2.5.0. A newer version may buring problem.

kioipp commented 6 years ago

@834721662 I have made sure that the protoc is 2.5.0 ,still thanks for reply . @junshi15 Is that any document mentioned about the OpenCV version ? I saw many install guides but non of them calling about this , If it's true then it's useless to build CaffeOnSpark because my DL lab used OpenCV 3.0+ as default environment .

JJJzheng commented 6 years ago

@kioipp I have encountered this situation that my cuda is 9.0 and my OpenCV is 3.0 and they are in conflict when make build. So I changed my OpenCV to 2.14.13 and my maven 3.5.0 to 3.2.5. Hope that can help you.

junshi15 commented 6 years ago

sorry, I think opencv 2.4 is what CaffeOnSpark supports. There are API changes between 2.4 and 3.2, hence the new openCV is not supported.