Closed Easonshow closed 1 year ago
The train log is below: Reducer buckets have been rebuilt in this iteration. iter: 100/180000, lr: 0.003454, eta: 5:22:45, time: 10.87, loss: 132873553.8683, loss_pre: 132873553.8683, loss_aux0: 0.0000, loss_aux1: 0.0000, loss_aux2: 0.0000, loss_aux3: 0.0000 iter: 200/180000, lr: 0.004348, eta: 5:12:08, time: 10.06, loss: 46347.5991, loss_pre: 46347.5991, loss_aux0: 0.0000, loss_aux1: 0.0000, loss_aux2: 0.0000, loss_aux3: 0.0000 iter: 300/180000, lr: 0.005474, eta: 5:12:48, time: 10.50, loss: 307194.5442, loss_pre: 307194.5442, loss_aux0: 0.0000, loss_aux1: 0.0000, loss_aux2: 0.0000, loss_aux3: 0.0000 iter: 400/180000, lr: 0.006892, eta: 5:08:07, time: 9.84, loss: 20168.8101, loss_pre: 20168.8101, loss_aux0: 0.0000, loss_aux1: 0.0000, loss_aux2: 0.0000, loss_aux3: 0.0000 iter: 500/180000, lr: 0.008676, eta: 5:08:08, time: 10.32, loss: 148601438.1153, loss_pre: 148601438.1153, loss_aux0: 0.0000, loss_aux1: 0.0000, loss_aux2: 0.0000, loss_aux3: 0.0000
The train log is below: Reducer buckets have been rebuilt in this iteration. iter: 100/180000, lr: 0.003454, eta: 5:22:45, time: 10.87, loss: 132873553.8683, loss_pre: 132873553.8683, loss_aux0: 0.0000, loss_aux1: 0.0000, loss_aux2: 0.0000, loss_aux3: 0.0000 iter: 200/180000, lr: 0.004348, eta: 5:12:08, time: 10.06, loss: 46347.5991, loss_pre: 46347.5991, loss_aux0: 0.0000, loss_aux1: 0.0000, loss_aux2: 0.0000, loss_aux3: 0.0000 iter: 300/180000, lr: 0.005474, eta: 5:12:48, time: 10.50, loss: 307194.5442, loss_pre: 307194.5442, loss_aux0: 0.0000, loss_aux1: 0.0000, loss_aux2: 0.0000, loss_aux3: 0.0000 iter: 400/180000, lr: 0.006892, eta: 5:08:07, time: 9.84, loss: 20168.8101, loss_pre: 20168.8101, loss_aux0: 0.0000, loss_aux1: 0.0000, loss_aux2: 0.0000, loss_aux3: 0.0000 iter: 500/180000, lr: 0.008676, eta: 5:08:08, time: 10.32, loss: 148601438.1153, loss_pre: 148601438.1153, loss_aux0: 0.0000, loss_aux1: 0.0000, loss_aux2: 0.0000, loss_aux3: 0.0000