search

ZongxuPan / DrBox-v2-tensorflow

The tensorflow implementation of DrBox-v2 which is an improved detector with rotatable boxes for target detection in remote sensing images
Apache License 2.0
47 stars 21 forks source link

Training process not stable #12

Open yzheng97 opened 5 years ago

yzheng97 commented 5 years ago

Durintg training, the loc_loss ocassionly became a very large value (As shown below). I checked that the calculation of loc_loss would not encounter the denominator=0 situation. I wonder why.

counter:[21000], loss:9098183573504.00000000, loc_loss:9098183573504.00000000, conf_loss:0.00971856, conf_pos_loss:0.00420601, conf_neg_loss:0.00551255, reg_loss:32.57967377

counter:[22000], loss:285012888911872.00000000, loc_loss:285012888911872.00000000, conf_loss:0.01419698, conf_pos_loss:0.00380920, conf_neg_loss:0.01038779, reg_loss:34.89893341

ZongxuPan commented 5 years ago

Durintg training, the loc_loss ocassionly became a very large value (As shown below). I checked that the calculation of loc_loss would not encounter the denominator=0 situation. I wonder why.

counter:[21000], loss:9098183573504.00000000, loc_loss:9098183573504.00000000, conf_loss:0.00971856, conf_pos_loss:0.00420601, conf_neg_loss:0.00551255, reg_loss:32.57967377

counter:[22000], loss:285012888911872.00000000, loc_loss:285012888911872.00000000, conf_loss:0.01419698, conf_pos_loss:0.00380920, conf_neg_loss:0.01038779, reg_loss:34.89893341

Since everyone using this code could encounter his/her specific issues, completely relying on us to solving these issues is unpractical, in part because the information is not enough in part because we have not so much time or effort, so we refer the users to the Q&A part in README, especially the second Q&A for your own data.