search

combust / mleap

MLeap: Deploy ML Pipelines to Production
https://combust.github.io/mleap-docs/
Apache License 2.0
1.5k stars 310 forks source link

Add support for spark-deep-learning (e.g. DeepImageFeaturizer) #537

Open FredYao opened 5 years ago

FredYao commented 5 years ago

Databricks has its own packge for running Deep Learning on Spark: https://github.com/databricks/spark-deep-learning Wondering if MLeap supports this or not? Thanks!

ancasarb commented 5 years ago

Hey @FredYao, no there's no direct support for the deep learning transformers under spark-deep-learning, but mleap does support tensorflow/keras if that is what you're looking after.

I'm happy to review a PR if you'd like to add them in, thank you!

FredYao commented 5 years ago

@ancasarb , thank you for your response. I was checking the mleap document (http://mleap-docs.combust.ml/) for TensorFlow support, however, I barely can find any examples or guidances for using that. Is there any code example that I can refer to? Thanks!

ancasarb commented 5 years ago

There are some TF examples in our unit tests https://github.com/combust/mleap/tree/master/mleap-tensorflow/src/test/scala/ml/combust/mleap/tensorflow. Please let me know if you have any questions.

Ben-Epstein commented 4 years ago

@ancasarb I don't see any examples of taking a Tensorflow graph created in Tensorflow/Keras and serializing it into MLeap.

The example you linked is Scala code that uses MLeaps custom built TensorflowModel object.

Does MLeap support taking a true Tensorflow/Keras model and serializing it? If so could you please provide an example of doing so?

Thank you!