Figure out distributed tensorflow

[tomowind]

TODO (read later):

• https://www.tensorflow.org/deploy/distributed
• https://gist.github.com/haje01/b655a9f0e4b6389b504d6a4e03dea379 (Korean)
• https://github.com/nlintz/TensorFlow-Tutorials/blob/master/10_save_restore_net.py
• https://github.com/tensorflow/models/tree/master/inception

[tomowind]

@oakcreek Googled with keras with distributed tensorflow, and found this: https://github.com/fchollet/keras/issues/1833

It seems keras doesn't use distributed tensorflow, and elephas seems the best option? To use distributed tensorflow, we need to write code in tensorflow.

[tomowind]

@oakcreek I was wrong. See Keras as a simplified interface to TensorFlow: tutorial. It's very simply done with 4 lines of code.

server = tf.train.Server.create_local_server()
sess = tf.Session(server.target)

from keras import backend as K
K.set_session(sess)