lehaifeng / MAPNet

40 stars 12 forks source link

How to know when should stop training in your code based on Loss and IoU output ? #5

Open SomayeKarimpour opened 3 years ago

SomayeKarimpour commented 3 years ago

Hi I did train the model and I got training loss and validation for each epoch among training like bellow: .... Epoch 150 |Iter 81200 |Loss 0.03061891 Epoch 150 |Iter 81300 |Loss 0.031306103 Epoch 150 |Iter 81400 |Loss 0.034883704 Epoch 150 |Iter 81500 |Loss 0.0326193 Epoch 150 |Iter 81600 |Loss 0.034357037 Time: 3870.109310865402 validate... last iou valu:0.65 new_iou value:0.9206227973963147 Save the checkpoint... Epoch 151 |Iter 81700 |Loss 0.03399411 Epoch 151 |Iter 81800 |Loss 0.030656744 Epoch 151 |Iter 81900 |Loss 0.033804014 Epoch 151 |Iter 82000 |Loss 0.034370854 Epoch 151 |Iter 82100 |Loss 0.033772305 Epoch 151 |Iter 82200 |Loss 0.03267585 Time: 315.97310042381287 validate... last iou valu:0.9206227973963147 new_iou value:0.9007333493933005 Epoch 152 |Iter 82300 |Loss 0.029079173 Epoch 152 |Iter 82400 |Loss 0.031596527 Epoch 152 |Iter 82500 |Loss 0.03290856 Epoch 152 |Iter 82600 |Loss 0.03253841 Epoch 152 |Iter 82700 |Loss 0.029811475 Time: 315.04660058021545 validate... last iou valu:0.9206227973963147 new_iou value:0.9206331145003502 ... How can I know when should I stop training? and how to know about overfitting?

liwenjielongren commented 3 years ago

image

liwenjielongren commented 3 years ago

my loss is negative number,why?