Open lijun20 opened 6 years ago
INFO:tensorflow:global step 6420: loss = 5.4209 (0.596 sec/step) INFO:tensorflow:global step 6430: loss = 5.3637 (0.334 sec/step) INFO:tensorflow:global step 6440: loss = 9.0471 (0.274 sec/step) INFO:tensorflow:global step 6450: loss = 6377.4712 (0.464 sec/step) INFO:tensorflow:global step 6460: loss = 5869.8906 (0.447 sec/step) INFO:tensorflow:global step 6470: loss = 1870.6338 (0.513 sec/step) INFO:tensorflow:global step 6480: loss = 711.0534 (0.274 sec/step) INFO:tensorflow:global step 6490: loss = 13.2399 (0.453 sec/step) INFO:tensorflow:global step 6500: loss = 8.3379 (0.381 sec/step) INFO:tensorflow:global step 6510: loss = 5.8521 (0.346 sec/step) ......... INFO:tensorflow:global step 10830: loss = 146732.0312 (0.392 sec/step) INFO:tensorflow:global step 10840: loss = 104813.1641 (0.487 sec/step) INFO:tensorflow:global step 10850: loss = 47788.4492 (0.803 sec/step) INFO:tensorflow:global step 10860: loss = 40910.5898 (0.389 sec/step) INFO:tensorflow:global step 10870: loss = 19970.1309 (0.450 sec/step) INFO:tensorflow:global step 10880: loss = 41234.4492 (0.281 sec/step) ......... INFO:tensorflow:global step 11480: loss = 3617.8975 (0.323 sec/step) INFO:tensorflow:global step 11490: loss = 13358.6592 (0.528 sec/step) INFO:tensorflow:global step 11500: loss = 22579.4043 (0.419 sec/step) INFO:tensorflow:global step 11510: loss = 16897972224.0000 (0.362 sec/step) INFO:tensorflow:global step 11520: loss = 18114068.0000 (0.509 sec/step) INFO:tensorflow:global step 11530: loss = 6474723.0000 (0.454 sec/step) INFO:tensorflow:global step 11540: loss = 231045.1250 (0.264 sec/step) INFO:tensorflow:global step 11620: loss = 186.8571 (0.364 sec/step) INFO:tensorflow:global step 11630: loss = 132.7600 (1.336 sec/step) INFO:tensorflow:global step 11640: loss = 89.9768 (0.917 sec/step)
The loss is too big and unstable. I try to reduce the learning rate, but I still can not solve it.
did you scale the coordinates between [0,1] ? What training parameters are you using? And what is the dataset you're training this model on?
@lijun20 你好,你后来解决这个问题了吗
INFO:tensorflow:global step 6420: loss = 5.4209 (0.596 sec/step) INFO:tensorflow:global step 6430: loss = 5.3637 (0.334 sec/step) INFO:tensorflow:global step 6440: loss = 9.0471 (0.274 sec/step) INFO:tensorflow:global step 6450: loss = 6377.4712 (0.464 sec/step) INFO:tensorflow:global step 6460: loss = 5869.8906 (0.447 sec/step) INFO:tensorflow:global step 6470: loss = 1870.6338 (0.513 sec/step) INFO:tensorflow:global step 6480: loss = 711.0534 (0.274 sec/step) INFO:tensorflow:global step 6490: loss = 13.2399 (0.453 sec/step) INFO:tensorflow:global step 6500: loss = 8.3379 (0.381 sec/step) INFO:tensorflow:global step 6510: loss = 5.8521 (0.346 sec/step) ......... INFO:tensorflow:global step 10830: loss = 146732.0312 (0.392 sec/step) INFO:tensorflow:global step 10840: loss = 104813.1641 (0.487 sec/step) INFO:tensorflow:global step 10850: loss = 47788.4492 (0.803 sec/step) INFO:tensorflow:global step 10860: loss = 40910.5898 (0.389 sec/step) INFO:tensorflow:global step 10870: loss = 19970.1309 (0.450 sec/step) INFO:tensorflow:global step 10880: loss = 41234.4492 (0.281 sec/step) ......... INFO:tensorflow:global step 11480: loss = 3617.8975 (0.323 sec/step) INFO:tensorflow:global step 11490: loss = 13358.6592 (0.528 sec/step) INFO:tensorflow:global step 11500: loss = 22579.4043 (0.419 sec/step) INFO:tensorflow:global step 11510: loss = 16897972224.0000 (0.362 sec/step) INFO:tensorflow:global step 11520: loss = 18114068.0000 (0.509 sec/step) INFO:tensorflow:global step 11530: loss = 6474723.0000 (0.454 sec/step) INFO:tensorflow:global step 11540: loss = 231045.1250 (0.264 sec/step) INFO:tensorflow:global step 11620: loss = 186.8571 (0.364 sec/step) INFO:tensorflow:global step 11630: loss = 132.7600 (1.336 sec/step) INFO:tensorflow:global step 11640: loss = 89.9768 (0.917 sec/step)
The loss is too big and unstable. I try to reduce the learning rate, but I still can not solve it.