amplab / SparkNet

Distributed Neural Networks for Spark
MIT License
603 stars 172 forks source link

build issue #136

Open jimht011 opened 8 years ago

jimht011 commented 8 years ago

Does anybody have build issue? We download the source code and one maven dependency is not available. imageio is not available and only imageio-core is available. When try to resolve dependency one by one, stuck at org.bytedeco.javacpp-presets - caffe . Not sure exactly version used in this project.

I think this is a very basic thing to let other users leverage this contribution.

pcmoritz commented 8 years ago

Have you tried using SBT? That is the way we are building the project. The maven file was contributed by @javadba and if it is currently broken, we should update or remove it.

javadba commented 8 years ago

Hi - the versions have changed in SparkNet and i will provide an update to the pom.xml by the coming weekend 7/4/16

pcmoritz commented 8 years ago

Great, thanks a lot!

javadba commented 8 years ago

Hi I am on Mac. I can not build due to the classifier for Mac are not built / stored on the amplab maven repo.

[ERROR] Failed to execute goal on project sparknet: Could not resolve dependencies for project org.amplab:sparknet:jar:1.0: The following artifacts could not be resolved: org.bytedeco.javacpp-presets:caffe:jar:macosx-x86_64:master-1.2-SPARKNET, org.bytedeco.javacpp-presets:opencv:jar:macosx-x86_64:3.1.0-1.2-SPARKNET, org.bytedeco.javacpp-presets:tensorflow:jar:macosx-x86_64:master-1.2-SPARKNET: Could not find artifact org.bytedeco.javacpp-presets:caffe:jar:macosx-x86_64:master-1.2-SPARKNET in javacpp (http://www.eecs.berkeley.edu/~rkn/snapshot-2016-03-05/) -> [Help 1]

javadba commented 8 years ago

So I have the updated pom.xml but can not test it.

javadba commented 8 years ago

OK it appears that the requirement were to build javacpp for mac locally: shown here;

https://github.com/bytedeco/javacpp-presets/wiki/Build-Environments

I will get back after attempting to use the locally built presets.

javadba commented 8 years ago

Wow. Building javacpp-presets is a big complicated deal. Is there any documentation you guys have about how to do it wrt SparkNet?

robertnishihara commented 8 years ago

There are some instructions here https://github.com/amplab/SparkNet/blob/master/doc/creating-jars.md

javadba commented 8 years ago

Those instructions do not work for mac/osx. I have been following the instructions for mac/osx from the javacpp-presets site.

After about an hour of compilations there is a failure on "tar: Unrecognized archive format". This is not a Sparknet project issue but a javacpp-presets project issue. Overall this is just a big job to get javacpp-presets up and running on os/x.

I am going to create a separate issue specifically for os/x: the issues here are not for my pom.xml or even spark 2.X .

javadba commented 8 years ago

This issue should be marked as depending on https://github.com/amplab/SparkNet/issues/139