Open gagamihuya opened 7 years ago
arundasan91
commented
7 years ago @gagamihuya , Do Caffe work with Protobuf 2.4.1 ?
The required version is 2.5.0 right ?
I am not really sure if version 2.4.1 works. I will check and let you know. By the way, if you follow the Caffe Installation Instruction to install the prerequisites then you'll install a protobuf version greater than 2.5.0 when you do for req in $(cat requirements.txt); do pip install $req; done inside the caffe/python folder.
arundasan91
commented
7 years ago The protobuf version I have locally is 2.5.0.
gagamihuya
commented
7 years ago Thinks very much, @ Arun Das
I have locally build success and all test can be runinig successfully. I want to use CaffeOnSpark on a huge cluster which protobuf is 2.4.1.
2017-01-18 10:00 GMT+08:00 Arun Das notifications@github.com:
The protobuf version I have locally is 2.5.0.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/yahoo/CaffeOnSpark/issues/222#issuecomment-273360733, or mute the thread https://github.com/notifications/unsubscribe-auth/AX2HnwNLhCMBJ3km_H09Pae1xVHcuZAYks5rTXJagaJpZM4LkUxh .
arundasan91
commented
7 years ago @gagamihuya , I can try to see if Protobuf V2.4.1 works but it will take some time as I am busy with some other work. I will let you know how it goes. Thanks.
junshi15
commented
7 years ago From the error message, it is likely some protobuf java api has changed from 2.4.1 to 2.5.0. I have not tried 2.4.1 myself, but if BLVC caffe works with protobuf 2.4.1, then you will need to modify the java side code, especially the following file. You may have to modify other places as well.
@arundasan91 How to build CaffeOnSpark with protobuf_2.4.1? [ERRPR-LOG] as below:
17/01/16 15:44:19 ERROR yarn.ApplicationMaster: User class threw exception: java.lang.VerifyError: class caffe.Caffe$SolverParameter overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet; java.lang.VerifyError: class caffe.Caffe$SolverParameter overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet; at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:760) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:455) at java.net.URLClassLoader.access$100(URLClassLoader.java:73) at java.net.URLClassLoader$1.run(URLClassLoader.java:367) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:360) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at com.yahoo.ml.jcaffe.Utils.GetSolverParam(Utils.java:13) at com.yahoo.ml.caffe.Config.protoFile_$eq(Config.scala:70) at com.yahoo.ml.caffe.Config.(Config.scala:439)
at com.yahoo.ml.caffe.CaffeOnSpark$.main(CaffeOnSpark.scala:34)
at com.yahoo.ml.caffe.CaffeOnSpark.main(CaffeOnSpark.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:542)
17/01/16 15:44:19 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.lang.VerifyError: class caffe.Caffe$SolverParameter overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;)
17/01/16 15:44:19 DEBUG yarn.ApplicationMaster: shutting down reporter thread