Viredery / tf-eager-fasterrcnn

Faster R-CNN R-101-FPN model was implemented with TensorFlow2.0 eager execution.
MIT License
89 stars 46 forks source link

Training return loss "nan" #26

Open ZeroSum0x00 opened 2 years ago

ZeroSum0x00 commented 2 years ago

I have loss error that returns "nan" when training from scratch without using model.load_weight, while if using load_weight it does not. I don't know why is that. Can you tell me before get your weights file, how did you train it? I look forward to your response very much. Thanks!

epoch: 0 , batch: 0 , loss: 2.7604368 epoch: 0 , batch: 100 , loss: nan Summary epoch: 0 , loss: nan epoch: 1 , batch: 0 , loss: nan epoch: 1 , batch: 100 , loss: nan Summary epoch: 1 , loss: nan epoch: 2 , batch: 0 , loss: nan epoch: 2 , batch: 100 , loss: nan Summary epoch: 2 , loss: nan