Does tensorflow_scala support distributed training

eaplatanios / tensorflow_scala

TensorFlow API for the Scala Programming Language

http://platanios.org/tensorflow_scala/

Apache License 2.0

939 stars 95 forks source link

Does tensorflow_scala support distributed training #174

Open algorithmdog opened 5 years ago

algorithmdog commented 5 years ago

I am using tensorflow_scala to train a model. The data is large and I want to use distributed training. Does tensorflow_scala support distributed training. If so, how can I get some documents and examples? Thanks in advance.

dwsmith1983 commented 4 years ago

@algorithmdog good question. It would great to be able to use this on a Spark cluster. I hope someone answers or looks into this since it hasn't been looked at in 4 months.

eaplatanios commented 4 years ago

Sorry for not responding earlier but I'm currently busy with multiple other projects and my graduation. In principle you should be able to implement a distributed trainer for your models relatively easily (especially if you're using data parallelism), but TF Scala does not provide any such pre-existing trainers.