yahoo / CaffeOnSpark

Distributed deep learning on Hadoop and Spark clusters.
Apache License 2.0
1.27k stars 358 forks source link

Error while "make build" --[caffe-grid Failure] #170

Open waynesuzq opened 7 years ago

waynesuzq commented 7 years ago

Dear,

I met a problem when make build in $CAFFE_ON_SPARK folder:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (compile-generator) on project caffe-distri: Compilation failure: Compilation failure:
[ERROR] /home/opuser/CaffeOnSpark/caffe-distri/src/main/java/caffe/Caffe.java:[39124,19] unmappable character for encoding ASCII

OS: Ubuntu VERSION="15.10 (Wily Werewolf) Followed the guide, I have already installed the caffe prerequists, and installed openblas separately. When I "make build", I met this problem. Details are as following:

[WARNING] File encoding has not been set, using platform encoding ANSI_X3.4-1968, i.e. build is platform dependent!
[INFO] Compiling 10 source files to /home/opuser/CaffeOnSpark/caffe-distri/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
...
/java/caffe/Caffe.java:[41274,21] unmappable character for encoding ASCII
[ERROR] /home/opuser/CaffeOnSpark/caffe-distri/src/main/java/caffe/Caffe.java:[41274,22] unmappable character for encoding ASCII
...
[ERROR] /home/opuser/CaffeOnSpark/caffe-distri/src/main/java/caffe/Caffe.java:[9919,31] cannot find symbol
  symbol:   class ProtocolStringList
  location: package com.google.protobuf
[ERROR] /home/opuser/CaffeOnSpark/caffe-distri/src/main/java/caffe/Caffe.java:[8280,24] cannot find symbol
  symbol:   class ProtocolStringList
  location: package com.google.protobuf
[ERROR] /home/opuser/CaffeOnSpark/caffe-distri/src/main/java/caffe/Caffe.java:[5761,31] cannot find symbol
  symbol:   class ProtocolStringList
  location: package com.google.protobuf
[ERROR] /home/opuser/CaffeOnSpark/caffe-distri/src/main/java/caffe/Caffe.java:[5238,24] cannot find symbol
  symbol:   class ProtocolStringList
  location: package com.google.protobuf
[ERROR] /home/opuser/CaffeOnSpark/caffe-distri/src/main/java/caffe/Caffe.java:[16820,31] cannot find symbol
  symbol:   class ProtocolStringList
  location: package com.google.protobuf

...
[ERROR] symbol:   method getUnmodifiableView()
[ERROR] location: variable top_ of type com.google.protobuf.LazyStringList
[ERROR] /home/opuser/CaffeOnSpark/caffe-distri/src/main/java/caffe/Caffe.java:[81043,22] cannot find symbol
[ERROR] symbol:   method getUnmodifiableView()
[ERROR] location: variable param_ of type com.google.protobuf.LazyStringList
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :caffe-distri
Makefile:19: recipe for target 'build' failed
make: *** [build] Error 1
waynesuzq commented 7 years ago

Hi, I'm running on ubuntu15, and I have succeeded making caffe, which means dependencies are not a problem.

protobuf-compiler: libprotoc 2.6.1 JAVA: java version "1.7.0_51" OpenJDK Runtime Environment (IcedTea 2.4.6) (7u51-2.4.6-1ubuntu4) OpenJDK 64-Bit Zero VM (build 24.51-b03, interpreted mode)

The problem has been solved by changing the protoc version in caffe-distri/pom.xml.

<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>2.6.1</version>
<scope>provided</scope>
</dependency>

However, an other error showed as below:

[INFO] Compiling 22 source files to /home/opuser/CaffeOnSpark/caffe-grid/target/classes at 1476967532207
[WARNING] warning: Class com.google.protobuf.ProtocolStringList not found - continuing with a stub.
[WARNING] warning: Class com.google.protobuf.ProtocolStringList not found - continuing with a stub.
[WARNING] warning: Class com.google.protobuf.ProtocolStringList not found - continuing with a stub.
...
[ERROR] error: Class com.google.protobuf.ProtocolStringList not found - continuing with a stub.
[ERROR] /home/opuser/CaffeOnSpark/caffe-grid/src/main/scala/com/yahoo/ml/caffe/ImageDataFrame.scala:44: error: applyDynamic does not support passing a vararg parameter
[INFO]       df = df.selectExpr(selects.toList:_*)
[INFO]                                  ^
[WARNING] 14 warnings found
[ERROR] two errors found
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] caffe .............................................. SUCCESS [  0.005 s]
[INFO] caffe-distri ....................................... SUCCESS [04:32 min]
[INFO] caffe-grid ......................................... FAILURE [03:14 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE

Anyone met this before?

junshi15 commented 7 years ago

CaffeOnSpark uses protobuf 2.5 API, which is incompatible with protobuf 2.6. Try down-grade protobuf to 2.5

bdgong commented 7 years ago

I meet similar problem like '.../main/java/caffe/Caffe.java:[9919,31] cannot find symbol symbol: class ProtocalStringList localtion: package com.google.protobuf ...' when build CaffeOnSpark on Ubuntu 16.04. However, how to downgrade protobuf from 2.6.1 to 2.5?