Maybe I do something wrong？current loss = nan

[tianwen0110]

Please fill out this issue template before submitting. Issues which do not fill out this template, or are already answered in the FAQs will simply be closed.

Please go to Stack Overflow for help and support. Also check past issues as many are repeats. Also check out the Frequently Asked Questions (FAQs) below in case your question has already been answered in an issue!

Issues should be one of the following:

1. Feature requests
2. Bug reports

---

Information

Please specify the following information when submitting an issue:

• What are your command line arguments?:
• python train.py --dataset person_part --continue_training True --batch_size 16 --crop_width 128 --crop_height 128 --h_flip false --v_flip false --model RefineNet --frontend ResNet101
• Have you written any custom code?: No
• What have you done to try and solve this issue?: Nothing
• TensorFlow version?: tensorflow_gpu 1.10.1

Describe the problem

Describe the problem clearly here. Be sure to convey here why it's a bug or a feature request.

I am trying to train on the Person-Part dataset but the current loss is always nan. I have already adjusted the class_dict.csv. From the first epoch, the average accuracy is very high: 0.998459, and it do not change whatever for which epoch.

[2019-02-18 15:10:46] Epoch = 0 Count = 80 Current_Loss = nan Time = 9.17 [2019-02-18 15:10:51] Epoch = 0 Count = 160 Current_Loss = nan Time = 5.15 [2019-02-18 15:10:57] Epoch = 0 Count = 240 Current_Loss = nan Time = 5.49 [2019-02-18 15:11:02] Epoch = 0 Count = 320 Current_Loss = nan Time = 5.26 [2019-02-18 15:11:08] Epoch = 0 Count = 400 Current_Loss = nan Time = 5.42

Average validation accuracy for epoch # 0000 = 0.998459 Average per class validation accuracies for epoch # 0000: head = 1.000000 torso = 0.800000 upper leg = 1.000000 upper arm = 0.800000 lower arm = 0.900000 lower leg = 1.000000 Validation precision = 1.0 Validation recall = 0.9984588623046875 Validation F1 score = 0.9992256053957942 Validation IoU score = 0.84122314453125

Average validation accuracy for epoch # 0010 = 0.998459 Average per class validation accuracies for epoch # 0010: head = 1.000000 torso = 0.800000 upper leg = 1.000000 upper arm = 0.800000 lower arm = 0.900000 lower leg = 1.000000 Validation precision = 1.0 Validation recall = 0.9984588623046875 Validation F1 score = 0.9992256053957942 Validation IoU score = 0.84122314453125

[developpeur3d]

I have the same problem , I modified only the load image function ( because I have a grayscale input images) Any idea how to solve this issue?

Unfortunatily, I see that there is no support on this repo .

[Shiny-ZhangXiXin]

I have the same question.I don't kown why one of the average class validation accuracies always be 1.00000. And before that, I also recived the wraning which is: UndefinedMetricWarning: Precision(and F-score) is ill-defined and being set to 0.0 in labels with no predicted samples.