Closed amughrabi closed 4 years ago
Instead of using CPU, I used GCloud based on the https://luminoth.readthedocs.io/en/latest/usage/cloud.html. It is running from 2 days ago.
I 2019-09-16T19:41:32.707196950Z master-replica-0 [master-0] - step: 138172, file: JPEGImages/cap036.jpg, train_loss: 1.38995575905, in 1.12s master-replica-0
I 2019-09-16T19:41:33.865154027Z master-replica-0 [master-0] - step: 138173, file: JPEGImages/cap624.jpg, train_loss: 1.34886920452, in 1.16s master-replica-0
Can you please advise how to know the remaining time for the training to be completed? am I falling in an infinite loop?
Can you please advise how to know the remaining time for the training to be completed? am I falling in an infinite loop?
I found a way to compute it!
B
, the average time spent for 1 step is S
, the number of epochs is 'P' in seconds.(P / B) * S
in seconds.
Hi team,
I would like to know if there is a way to know the estimated time for the
lumi train -c config.yaml
method. AFAICT, I created a dataset contains 7809 images (coco style), and the config.yaml isI run the training on CPU (
Intel(R) Core(TM) i7-7600U CPU @ 2.80GHz
), and here is a sample outputIt is running before 4 days ago, is this expected and how can I know how much remaining (if possible)?
Thanks a million, Mughrabi