CaffeOnSpark

[bhack]

What will be the impact of Yahoo CaffeOnSpark release for SparkNet?

[robertnishihara]

Great question! @anfeng, @junshi15, Mridul Jain, and @cypof are doing some really great work with CaffeOnSpark.

Long term, this will depend on adoption. In the meantime, there are enough differences in design decisions and implementation choices between the two projects that it makes sense to develop both.

[bhack]

...And also dl4j. Are there any plan to enter in spark-ml? Kudos to @saudet for his Caffe and Tensorflow java-cpp preset.

[robertnishihara]

Deeplearning4j is a very different and interesting approach to deep networks on Spark. On the one hand, I would expect it to work more seamlessly since they build everything themselves and so have to worry less about integrating with existing open source projects and native code. On the other hand, compatibility with existing standards (Caffe models and TensorFlow models) is harder. It's a good idea and very worthwhile, but it doesn't change our immediate plans.

As for Spark-ML. We are open to the idea, but haven't started working on that.

Indeed kudos to @saudet. He's doing an amazing job with JavaCPP.

[bhack]

Related.. https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/distributed_runtime/README.md

[agibsonccc]

Should have cced me ;) thanks @saudet. We use data parallelism in our models and integrate at the slave level. We try to keep the matrix computations self contained via our matrix lib nd4j (numpy ported to java with hardware accel)

We just added a computation graph api as well. We intend on adding other neural nets via javacpp from here on out as well. FWIW, we are starting to work with IBM on some of their integration /cc @cfregly if that helps at all.

Adding backends to layers is one of our next priorities which will mean different kinds of integrations. That will play well with the computation graph.

I think there's plenty of integration points if you guys are open for discussion.

FWIW, we are almost done with our internal rewrite for c++ internals which will allow us to integrate a bit smoother with different hardware. I'm happy to talk about adding other c++ in there as well.

Next will be adding a parameter server since there hasn't been interest on the spark side. FWIW I'm based in the bay area if an in person meeting is required. Good job on spark net!

[robertnishihara]

Hi @agibsonccc, and thanks for commenting! Sounds like you guys have a lot of great stuff in the pipeline. We're also based in the bay area and looking forward to discussing more!

[bhack]

@robertnishihara Are you in contact with @thunterdb? Is there also an independent roadmap for Spark-Tensorflow by Databricks? I really hope the there are enough energies to not let fragmentation overcome.

[robertnishihara]

We've talked, but haven't coordinated. Thanks for bringing this up! @thunterdb, we're also interested in your Spark-Tensorflow roadmap.

[bhack]

I suggest to take a look also to https://github.com/maxpumperla/elephas. Elephas rely on Keras that already interface Theano and Tensorflow backends via Python. I know that Keras has an easy path to integrate than Java but could be interesting to know other multi backend approach now that Google started to released the "official" distributed version of Tensorflow.

[robertnishihara]

Thanks for the link!

[agibsonccc]

I'll reach out. On Feb 28, 2016 10:41 AM, "Robert Nishihara" notifications@github.com wrote:

Thanks for the link!

— Reply to this email directly or view it on GitHub https://github.com/amplab/SparkNet/issues/96#issuecomment-189920304.

[bhack]

/cc @maxpumperla