search

cerndb / dist-keras

Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.
http://joerihermans.com/work/distributed-keras/
GNU General Public License v3.0
623 stars 169 forks source link

Progress Bar #38

Closed raviolli closed 6 years ago

raviolli commented 6 years ago

Hi, I've been using the dist-keras and its been great so far, I've encountered a problem though for large datasets (eg:// 1,000,000 training rows) it can take > 12h. Is there a progress bar like with keras.

Iteration 1 Epoch 1/1 4500/80460 [=>.............................................] - ETA: 795s - loss 3.5021

JoeriHermans commented 6 years ago

Hi,

There is no progress bar in dist-keras. However, you can view the progress of your job indirectly by viewing the number of processed rows using the Spark metrics.

However, it should be possible to implement a progress-bar.

Joeri