dahaichangge
commented
7 years ago I used CPU model ,and I did as #112 , and I comment out all the calls to Caffe.set_mode(Caffe.GPU) in the CifarApp,but is did not work.
Open dahaichangge opened 7 years ago
dahaichangge
commented
7 years ago I used CPU model ,and I did as #112 , and I comment out all the calls to Caffe.set_mode(Caffe.GPU) in the CifarApp,but is did not work.
dahaichangge
commented
7 years ago when I put the libcaffe.so.1.0.0-rc3 libjnicaffe.so into the path of jdk , and I put the jdk path into the LD_LIBRARY_PATH
export LD_LIBRARY_PATH=${JAVA_HOME}/lib:${JRE_HOME}/lib:/home/liutao/local_install/lib:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=/usr/local/cuda-7.5/targets/x86_64-linux/lib:$LD_LIBRARY_PATH
mistakes changed is shown below.
16/10/05 10:29:18 INFO spark.ContextCleaner: Cleaned accumulator 7
16/10/05 10:29:18 INFO spark.ContextCleaner: Cleaned accumulator 6
16/10/05 10:29:18 INFO spark.ContextCleaner: Cleaned accumulator 5
16/10/05 10:29:18 INFO spark.ContextCleaner: Cleaned accumulator 4
16/10/05 10:29:20 ERROR executor.Executor: Exception in task 0.0 in stage 12.0 (TID 10)
java.lang.UnsatisfiedLinkError: /usr/jdk/lib/libjnicaffe.so: /usr/jdk/lib/libjnicaffe.so: undefined symbol: _ZN5caffe15WindowDataLayerIdED1Ev
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1880)
at java.lang.Runtime.loadLibrary0(Runtime.java:849)
at java.lang.System.loadLibrary(System.java:1088)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:632)
at org.bytedeco.javacpp.Loader.load(Loader.java:470)
at org.bytedeco.javacpp.Loader.load(Loader.java:407)
at org.bytedeco.javacpp.caffe.
16/10/05 10:29:20 ERROR scheduler.TaskSetManager: Task 0 in stage 12.0 failed 1 times; aborting job
16/10/05 10:29:20 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 12.0, whose tasks have all completed, from pool
16/10/05 10:29:20 INFO scheduler.TaskSchedulerImpl: Cancelling stage 12
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/static/sql,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL/execution/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL/execution,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/metrics/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage/kill,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/api,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/static,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors/threadDump/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors/threadDump,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/environment/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/environment,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage/rdd/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage/rdd,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/pool/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/pool,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs/job/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs/job,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs/json,null}
16/10/05 10:29:20 INFO handler.ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs,null}
16/10/05 10:29:20 INFO scheduler.DAGScheduler: ResultStage 12 (foreach at CifarApp.scala:82) failed in 4.206 s
16/10/05 10:29:20 INFO scheduler.DAGScheduler: Job 6 failed: foreach at CifarApp.scala:82, took 4.224218 s
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 12.0 failed 1 times, most recent failure: Lost task 0.0 in stage 12.0 (TID 10, localhost): java.lang.UnsatisfiedLinkError: /usr/jdk/lib/libjnicaffe.so: /usr/jdk/lib/libjnicaffe.so: undefined symbol: _ZN5caffe15WindowDataLayerIdED1Ev
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1880)
at java.lang.Runtime.loadLibrary0(Runtime.java:849)
at java.lang.System.loadLibrary(System.java:1088)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:632)
at org.bytedeco.javacpp.Loader.load(Loader.java:470)
at org.bytedeco.javacpp.Loader.load(Loader.java:407)
at org.bytedeco.javacpp.caffe.
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at scala.Option.foreach(Option.scala:236)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1832)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1845)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1858)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1929)
at org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:912)
at org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:910)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
at org.apache.spark.rdd.RDD.foreach(RDD.scala:910)
at apps.CifarApp$.main(CifarApp.scala:82)
at apps.CifarApp.main(CifarApp.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.UnsatisfiedLinkError: /usr/jdk/lib/libjnicaffe.so: /usr/jdk/lib/libjnicaffe.so: undefined symbol: _ZN5caffe15WindowDataLayerIdED1Ev
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1880)
at java.lang.Runtime.loadLibrary0(Runtime.java:849)
at java.lang.System.loadLibrary(System.java:1088)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:632)
at org.bytedeco.javacpp.Loader.load(Loader.java:470)
at org.bytedeco.javacpp.Loader.load(Loader.java:407)
at org.bytedeco.javacpp.caffe.
Any help would be greatly appreciated! Many thanks in advance.
I run it on the virtual machine:Centos7, hadoop2.6.0, spark1.6.0, cuda-7.5. These mistakes were encountered is shown below. Any help would be greatly appreciated! Many thanks in advance.
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 12.0 failed 1 times, most recent failure: Lost task 0.0 in stage 12.0 (TID 34, localhost): java.lang.UnsatisfiedLinkError: no jnicaffe in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886) at java.lang.Runtime.loadLibrary0(Runtime.java:849) at java.lang.System.loadLibrary(System.java:1088) at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:632) at org.bytedeco.javacpp.Loader.load(Loader.java:470) at org.bytedeco.javacpp.Loader.load(Loader.java:407) at org.bytedeco.javacpp.caffe.(caffe.java:16)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at org.bytedeco.javacpp.Loader.load(Loader.java:442)
at org.bytedeco.javacpp.Loader.load(Loader.java:407)
at org.bytedeco.javacpp.caffe$NetParameter.(caffe.java:1946)
at apps.CifarApp$$anonfun$main$3.apply$mcVI$sp(CifarApp.scala:83)
at apps.CifarApp$$anonfun$main$3.apply(CifarApp.scala:82)
at apps.CifarApp$$anonfun$main$3.apply(CifarApp.scala:82)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at org.apache.spark.InterruptibleIterator.foreach(InterruptibleIterator.scala:28)
at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$32.apply(RDD.scala:912)
at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$32.apply(RDD.scala:912)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.UnsatisfiedLinkError: /tmp/javacpp378041962097/libjnicaffe.so: /tmp/javacpp378041962097/libjnicaffe.so: undefined symbol: _ZN5caffe15WindowDataLayerIdED1Ev
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
at java.lang.Runtime.load0(Runtime.java:795)
at java.lang.System.load(System.java:1062)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:615)
... 23 more
Driver stacktrace: at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419) at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799) at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799) at scala.Option.foreach(Option.scala:236) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588) at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1832) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1845) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1858) at org.apache.spark.SparkContext.runJob(SparkContext.scala:1929) at org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:912) at org.apache.spark.rdd.RDD$$anonfun$foreach$1.apply(RDD.scala:910) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111) at org.apache.spark.rdd.RDD.withScope(RDD.scala:316) at org.apache.spark.rdd.RDD.foreach(RDD.scala:910) at apps.CifarApp$.main(CifarApp.scala:82) at apps.CifarApp.main(CifarApp.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.UnsatisfiedLinkError: no jnicaffe in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886) at java.lang.Runtime.loadLibrary0(Runtime.java:849) at java.lang.System.loadLibrary(System.java:1088) at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:632) at org.bytedeco.javacpp.Loader.load(Loader.java:470) at org.bytedeco.javacpp.Loader.load(Loader.java:407) at org.bytedeco.javacpp.caffe.(caffe.java:16)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:274)
at org.bytedeco.javacpp.Loader.load(Loader.java:442)
at org.bytedeco.javacpp.Loader.load(Loader.java:407)
at org.bytedeco.javacpp.caffe$NetParameter.(caffe.java:1946)
at apps.CifarApp$$anonfun$main$3.apply$mcVI$sp(CifarApp.scala:83)
at apps.CifarApp$$anonfun$main$3.apply(CifarApp.scala:82)
at apps.CifarApp$$anonfun$main$3.apply(CifarApp.scala:82)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at org.apache.spark.InterruptibleIterator.foreach(InterruptibleIterator.scala:28)
at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$32.apply(RDD.scala:912)
at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$32.apply(RDD.scala:912)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.UnsatisfiedLinkError: /tmp/javacpp378041962097/libjnicaffe.so: /tmp/javacpp378041962097/libjnicaffe.so: undefined symbol: _ZN5caffe15WindowDataLayerIdED1Ev
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
at java.lang.Runtime.load0(Runtime.java:795)
at java.lang.System.load(System.java:1062)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:615)
... 23 more
16/10/04 09:15:27 INFO ui.SparkUI: Stopped Spark web UI at http://192.168.116.136:4040
16/10/04 09:15:27 INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
16/10/04 09:15:28 INFO storage.MemoryStore: MemoryStore cleared
16/10/04 09:15:28 INFO storage.BlockManager: BlockManager stopped
16/10/04 09:15:28 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
16/10/04 09:15:28 INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
16/10/04 09:15:28 INFO spark.SparkContext: Successfully stopped SparkContext
16/10/04 09:15:28 INFO util.ShutdownHookManager: Shutdown hook called
16/10/04 09:15:28 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-66f96f4b-4f6e-4600-9399-6caa0ac1aa46
16/10/04 09:15:28 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-66f96f4b-4f6e-4600-9399-6caa0ac1aa46/httpd-85eb2e4d-bcb9-43de-9f44-5e936dc3f9ec
[root@master bin]#