Thanks a lot for implementing ENAS in Pytorch! I am able to run ENAS with Penn Treebank dataset. I am trying to understand the output during training. At beginning, the output shows some information about the gradients, for example:
2018-08-29 23:53:50,910:INFO::abs max grad 0.5459082126617432
2018-08-29 23:53:56,523:INFO::abs max grad 0.569364070892334
2018-08-29 23:54:02,990:INFO::abs max grad 0.6024199724197388
2018-08-29 23:54:11,814:INFO::max hidden 16.342870712280273
Then it shows loss and ppl but the epoch number is 0 all the time.
2018-08-29 23:54:17,591:INFO::| epoch 0 | lr 20.00 | raw loss 8.33 | loss 8.33 | ppl 4140.68
2018-08-29 23:54:58,164:INFO::| epoch 0 | lr 20.00 | raw loss 7.29 | loss 7.29 | ppl 1470.41
But after a few epochs, when ppl goes down to 600~, the output changes to show loss and entropy. No ppl is shown.
2018-08-30 00:03:58,605:INFO::| epoch 0 | lr 0.00035 | R 0.13311 | entropy 1.1463 | loss 0.00677
2018-08-30 00:04:10,912:INFO::| epoch 0 | lr 0.00035 | R 0.13406 | entropy 1.1326 | loss 0.01401
Could anyone help me understand the output a little? Why does the output stop showing ppl and why the epoch number is always 0? How to directly monitor the ppl?
Hey,
Thanks a lot for implementing ENAS in Pytorch! I am able to run ENAS with Penn Treebank dataset. I am trying to understand the output during training. At beginning, the output shows some information about the gradients, for example:
2018-08-29 23:53:50,910:INFO::abs max grad 0.5459082126617432 2018-08-29 23:53:56,523:INFO::abs max grad 0.569364070892334 2018-08-29 23:54:02,990:INFO::abs max grad 0.6024199724197388 2018-08-29 23:54:11,814:INFO::max hidden 16.342870712280273
Then it shows loss and ppl but the epoch number is 0 all the time. 2018-08-29 23:54:17,591:INFO::| epoch 0 | lr 20.00 | raw loss 8.33 | loss 8.33 | ppl 4140.68 2018-08-29 23:54:58,164:INFO::| epoch 0 | lr 20.00 | raw loss 7.29 | loss 7.29 | ppl 1470.41
But after a few epochs, when ppl goes down to 600~, the output changes to show loss and entropy. No ppl is shown. 2018-08-30 00:03:58,605:INFO::| epoch 0 | lr 0.00035 | R 0.13311 | entropy 1.1463 | loss 0.00677 2018-08-30 00:04:10,912:INFO::| epoch 0 | lr 0.00035 | R 0.13406 | entropy 1.1326 | loss 0.01401
Could anyone help me understand the output a little? Why does the output stop showing ppl and why the epoch number is always 0? How to directly monitor the ppl?
Thanks! Yue