amplab / SparkNet

Distributed Neural Networks for Spark
MIT License
603 stars 172 forks source link

Current Spark Net cannot build for both CPU and GPU #143

Open zenonlpc opened 8 years ago

zenonlpc commented 8 years ago

Hello guys

I followed the instructions here https://github.com/amplab/SparkNet to build my own AMI

When building sparknet using command "sbt assembly" I got this error information:

[warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: UNRESOLVED DEPENDENCIES :: [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] :: org.bytedeco#javacpp;1.2-SPARKNET: not found [warn] :: org.bytedeco.javacpp-presets#caffe;master-1.2-SPARKNET: not found [warn] :: org.bytedeco.javacpp-presets#opencv;3.1.0-1.2-SPARKNET: not found [warn] :: org.bytedeco.javacpp-presets#tensorflow;master-1.2-SPARKNET: not found [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] [warn] Note: Unresolved dependencies path: [warn] org.bytedeco:javacpp:1.2-SPARKNET (/home/ubuntu/SparkNet/build.sbt#L11-12) [warn] +- default:sparknet_2.10:0.1-SNAPSHOT [warn] org.bytedeco.javacpp-presets:caffe:master-1.2-SPARKNET (/home/ubuntu/SparkNet/build.sbt#L13-14) [warn] +- default:sparknet_2.10:0.1-SNAPSHOT [warn] org.bytedeco.javacpp-presets:opencv:3.1.0-1.2-SPARKNET (/home/ubuntu/SparkNet/build.sbt#L17-18) [warn] +- default:sparknet_2.10:0.1-SNAPSHOT [warn] org.bytedeco.javacpp-presets:tensorflow:master-1.2-SPARKNET (/home/ubuntu/SparkNet/build.sbt#L21-22) [warn] +- default:sparknet_2.10:0.1-SNAPSHOT sbt.ResolveException: unresolved dependency: org.bytedeco#javacpp;1.2-SPARKNET: not found unresolved dependency: org.bytedeco.javacpp-presets#caffe;master-1.2-SPARKNET: not found unresolved dependency: org.bytedeco.javacpp-presets#opencv;3.1.0-1.2-SPARKNET: not found unresolved dependency: org.bytedeco.javacpp-presets#tensorflow;master-1.2-SPARKNET: not found at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:313) at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:191) at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:168) at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156) at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:156) at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:133) at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57) at sbt.IvySbt$$anon$4.call(Ivy.scala:65) at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93) at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78) at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97) at xsbt.boot.Using$.withResource(Using.scala:10) at xsbt.boot.Using$.apply(Using.scala:9) at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58) at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48) at xsbt.boot.Locks$.apply0(Locks.scala:31) at xsbt.boot.Locks$.apply(Locks.scala:28) at sbt.IvySbt.withDefaultLogger(Ivy.scala:65) at sbt.IvySbt.withIvy(Ivy.scala:128) at sbt.IvySbt.withIvy(Ivy.scala:125) at sbt.IvySbt$Module.withModule(Ivy.scala:156) at sbt.IvyActions$.updateEither(IvyActions.scala:168) at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1442) at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1438) at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$90.apply(Defaults.scala:1473) at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$90.apply(Defaults.scala:1471) at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:37) at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1476) at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1470) at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:60) at sbt.Classpaths$.cachedUpdate(Defaults.scala:1493) at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1420) at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1372) at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47) at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40) at sbt.std.Transform$$anon$4.work(System.scala:63) at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228) at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228) at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17) at sbt.Execute.work(Execute.scala:237) at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228) at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228) at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159) at sbt.CompletionService$$anon$2.call(CompletionService.scala:28) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) error sbt.ResolveException: unresolved dependency: org.bytedeco#javacpp;1.2-SPARKNET: not found [error] unresolved dependency: org.bytedeco.javacpp-presets#caffe;master-1.2-SPARKNET: not found [error] unresolved dependency: org.bytedeco.javacpp-presets#opencv;3.1.0-1.2-SPARKNET: not found [error] unresolved dependency: org.bytedeco.javacpp-presets#tensorflow;master-1.2-SPARKNET: not found [error] Total time: 27 s, completed Sep 6, 2016 3:46:51 PM

After that I also tried building CPU from this location: http://www.eecs.berkeley.edu/~rkn/snapshot-2016-03-16-CPU/

I got same error instead SPARKNET not found, it says SPARKNETCPU not found.

Please advise!

Thanks in advance!

Pengcheng

rachmaninovquartet commented 7 years ago

Hi, I'm having the same issue. Did you figure out a workaround yet?

zenonlpc commented 7 years ago

Hi

I haven't figured out a way to build it.

On Mon, Sep 19, 2016 at 3:50 PM, rachmaninovquartet < notifications@github.com> wrote:

Hi, I'm having the same issue. Did you figure out a workaround yet?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/amplab/SparkNet/issues/143#issuecomment-248103180, or mute the thread https://github.com/notifications/unsubscribe-auth/AR1eZF98N5KZ6DoqGpXkhyxHLRycivt9ks5qrueTgaJpZM4J2AQp .

hckuo2 commented 7 years ago

+1

jondeanhaley commented 7 years ago

+1

saman-aghazadeh commented 7 years ago

+1

rawar-zz commented 7 years ago

+1