search

amplab / SparkNet

Distributed Neural Networks for Spark
MIT License
603 stars 172 forks source link

sbt assembly failed in Ubuntu 14.04 #144

Open apollos opened 7 years ago

apollos commented 7 years ago

After clone the repo, run sbt assembly, I get the followed error.

16/09/13 22:29:58 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down. [info] NDArraySpec: [info] TensorFlowNetSpec: [info] GraphDef java.lang.UnsatisfiedLinkError: no jnitensorflow in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:632) at org.bytedeco.javacpp.Loader.load(Loader.java:470) at org.bytedeco.javacpp.Loader.load(Loader.java:407) at org.bytedeco.javacpp.tensorflow.(tensorflow.java:10) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.bytedeco.javacpp.Loader.load(Loader.java:442) at org.bytedeco.javacpp.Loader.load(Loader.java:407) at org.bytedeco.javacpp.tensorflow$GraphDef.(tensorflow.java:6760) at TensorFlowNetSpec$$anonfun$1.apply$mcV$sp(TensorFlowNetSpec.scala:15) at TensorFlowNetSpec$$anonfun$1.apply(TensorFlowNetSpec.scala:14) at TensorFlowNetSpec$$anonfun$1.apply(TensorFlowNetSpec.scala:14) at org.scalatest.Transformer$$anonfun$apply$1.apply(Transformer.scala:22) at org.scalatest.Transformer$$anonfun$apply$1.apply(Transformer.scala:22) at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85) at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) at org.scalatest.Transformer.apply(Transformer.scala:22) at org.scalatest.Transformer.apply(Transformer.scala:20) at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1636) at org.scalatest.Suite$class.withFixture(Suite.scala:1121) at org.scalatest.FlatSpec.withFixture(FlatSpec.scala:1683) at org.scalatest.FlatSpecLike$class.invokeWithFixture$1(FlatSpecLike.scala:1633) at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1645) at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1645) at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306) at org.scalatest.FlatSpecLike$class.runTest(FlatSpecLike.scala:1645) at org.scalatest.FlatSpec.runTest(FlatSpec.scala:1683) at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1703) at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1703) at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413) at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401) at scala.collection.immutable.List.foreach(List.scala:318) at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401) at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:390) at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:427) at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401) at scala.collection.immutable.List.foreach(List.scala:318) at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401) at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396) at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483) at org.scalatest.FlatSpecLike$class.runTests(FlatSpecLike.scala:1703) at org.scalatest.FlatSpec.runTests(FlatSpec.scala:1683) at org.scalatest.Suite$class.run(Suite.scala:1423) at org.scalatest.FlatSpec.org$scalatest$FlatSpecLike$$super$run(FlatSpec.scala:1683) at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1749) at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1749) at org.scalatest.SuperEngine.runImpl(Engine.scala:545) at org.scalatest.FlatSpecLike$class.run(FlatSpecLike.scala:1749) at org.scalatest.FlatSpec.run(FlatSpec.scala:1683) at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:444) at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:651) at sbt.TestRunner.runTest$1(TestFramework.scala:76) at sbt.TestRunner.run(TestFramework.scala:85) at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202) at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202) at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:185) at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202) at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202) at sbt.TestFunction.apply(TestFramework.scala:207) at sbt.Tests$.sbt$Tests$$processRunnable$1(Tests.scala:239) at sbt.Tests$$anonfun$makeSerial$1.apply(Tests.scala:245) at sbt.Tests$$anonfun$makeSerial$1.apply(Tests.scala:245) at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44) at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44) at sbt.std.Transform$$anon$4.work(System.scala:63) at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228) at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228) at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17) at sbt.Execute.work(Execute.scala:237) at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228) at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228) at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159) at sbt.CompletionService$$anon$2.call(CompletionService.scala:28) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.UnsatisfiedLinkError: /tmp/javacpp4596648265763/libjnitensorflow.so: libcudart.so.7.0: cannot open shared object file: No such file or directory at java.lang.ClassLoader$NativeLibrary.load(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) at java.lang.Runtime.load0(Runtime.java:809) at java.lang.System.load(System.java:1086) at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:615) at org.bytedeco.javacpp.Loader.load(Loader.java:470) at org.bytedeco.javacpp.Loader.load(Loader.java:407) at org.bytedeco.javacpp.tensorflow.(tensorflow.java:10) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.bytedeco.javacpp.Loader.load(Loader.java:442) at org.bytedeco.javacpp.Loader.load(Loader.java:407) at org.bytedeco.javacpp.tensorflow$GraphDef.(tensorflow.java:6760) at TensorFlowNetSpec$$anonfun$1.apply$mcV$sp(TensorFlowNetSpec.scala:15) at TensorFlowNetSpec$$anonfun$1.apply(TensorFlowNetSpec.scala:14) at TensorFlowNetSpec$$anonfun$1.apply(TensorFlowNetSpec.scala:14) at org.scalatest.Transformer$$anonfun$apply$1.apply(Transformer.scala:22) at org.scalatest.Transformer$$anonfun$apply$1.apply(Transformer.scala:22) at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85) at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) at org.scalatest.Transformer.apply(Transformer.scala:22) at org.scalatest.Transformer.apply(Transformer.scala:20) at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1636) at org.scalatest.Suite$class.withFixture(Suite.scala:1121) at org.scalatest.FlatSpec.withFixture(FlatSpec.scala:1683) at org.scalatest.FlatSpecLike$class.invokeWithFixture$1(FlatSpecLike.scala:1633) at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1645) at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1645) at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306) at org.scalatest.FlatSpecLike$class.runTest(FlatSpecLike.scala:1645) at org.scalatest.FlatSpec.runTest(FlatSpec.scala:1683) at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1703) at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1703) at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413) at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401) at scala.collection.immutable.List.foreach(List.scala:318) at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401) at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:390) at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:427) at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401) at scala.collection.immutable.List.foreach(List.scala:318) at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401) at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396) at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483) at org.scalatest.FlatSpecLike$class.runTests(FlatSpecLike.scala:1703) at org.scalatest.FlatSpec.runTests(FlatSpec.scala:1683) at org.scalatest.Suite$class.run(Suite.scala:1423) at org.scalatest.FlatSpec.org$scalatest$FlatSpecLike$$super$run(FlatSpec.scala:1683) at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1749) at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1749) at org.scalatest.SuperEngine.runImpl(Engine.scala:545) at org.scalatest.FlatSpecLike$class.run(FlatSpecLike.scala:1749) at org.scalatest.FlatSpec.run(FlatSpec.scala:1683) at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:444) at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:651) at sbt.TestRunner.runTest$1(TestFramework.scala:76) at sbt.TestRunner.run(TestFramework.scala:85) at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202) at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202) at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:185) at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202) at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202) at sbt.TestFunction.apply(TestFramework.scala:207) at sbt.Tests$.sbt$Tests$$processRunnable$1(Tests.scala:239) at sbt.Tests$$anonfun$makeSerial$1.apply(Tests.scala:245) at sbt.Tests$$anonfun$makeSerial$1.apply(Tests.scala:245) at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44) at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44) at sbt.std.Transform$$anon$4.work(System.scala:63) at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228) at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228) at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17) at sbt.Execute.work(Execute.scala:237) at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228) at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228) at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159) at sbt.CompletionService$$anon$2.call(CompletionService.scala:28) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) [error] Could not run test TensorFlowNetSpec: java.lang.UnsatisfiedLinkError: no jnitensorflow in java.library.path [info] LoadAdultDataSpec: [info] - should be able to load the adult dataset !!! IGNORED !!! [info] Run completed in 6 seconds, 606 milliseconds. [info] Total number of tests run: 6 [info] Suites: completed 5, aborted 0 [info] Tests: succeeded 6, failed 0, canceled 0, ignored 9, pending 0 [info] All tests passed. [error] Error during tests: [error] TensorFlowNetSpec error sbt.TestsFailedException: Tests unsuccessful [error] Total time: 7 s, completed Sep 13, 2016 10:30:01 PM

ckchow commented 7 years ago

What does your build.sbt look like? It seems like the dependency on tensorflow-javacpp stuff could be being pulled incorrectly.

apollos commented 7 years ago

Yes, I also doubt it but I do not change the build.sbt

21 libraryDependencies += "org.bytedeco.javacpp-presets" % "tensorflow" % "master-1.2-SPARKNET" 22 23 libraryDependencies += "org.bytedeco.javacpp-presets" % "tensorflow" % "master-1.2-SPARKNET" classifier "linux-x86_64"

ckchow commented 7 years ago

I am referring specifically to the resolvers line - I think it starts like (at line 9)

resolvers += "javacpp" at "http://www.eecs.berkeley.edu/~rkn/snapshot-2016-03-05/"

but you should change it to

resolvers += "javacpp" at "http://people.eecs.berkeley.edu/~rkn/snapshot-2016-03-05/"