search

dragen1860 / TensorFlow-2.x-Tutorials

TensorFlow 2.x version's Tutorials and Examples, including CNN, RNN, GAN, Auto-Encoders, FasterRCNN, GPT, BERT examples, etc. TF 2.0版入门实例代码，实战教程。
6.38k stars 2.24k forks source link

Gradients do not exist for variables #39

Open CanshangD opened 4 years ago

CanshangD commented 4 years ago

I have got the error “Gradients do not exist for variables” when training faster rcnn， and the loss is nan.

log： W1108 09:59:39.735346 139647501481728 optimizer_v2.py:1029] Gradients do not exist for variables ['faster_rcnn/b_box_head/rcnn_class_conv1/kernel:0', 'faster_rcnn/b_box_head/rcnn_class_conv1/bias:0', 'faster_rcnn/b_box_head/rcnn_class_bn1/gamma:0', 'faster_rcnn/b_box_head/rcnn_class_bn1/beta:0', 'faster_rcnn/b_box_head/rcnn_class_conv2/kernel:0', 'faster_rcnn/b_box_head/rcnn_class_conv2/bias:0', 'faster_rcnn/b_box_head/rcnn_class_bn2/gamma:0', 'faster_rcnn/b_box_head/rcnn_class_bn2/beta:0', 'faster_rcnn/b_box_head/rcnn_class_logits/kernel:0', 'faster_rcnn/b_box_head/rcnn_class_logits/bias:0', 'faster_rcnn/b_box_head/rcnn_bbox_fc/kernel:0', 'faster_rcnn/b_box_head/rcnn_bbox_fc/bias:0'] when minimizing the loss. epoch 0 2 nan

MLForNerds commented 4 years ago

Got the same issue, couldn't solve it. Kindly post the solution if you have solved it.

IngLP commented 4 years ago

I have the same issue and for me too the loss is NaN. Please help

xiaolng commented 3 years ago

same issue, loss becomes nan

flori-shan commented 2 years ago

same issue, please help

jgarza9788 commented 2 years ago

can you please say what file you were running when you got this error? what version(s) of pytorch and python etc ?