评估S-measure时候出现Nan错误

thb1314 commented 5 years ago

您好，Q.item()返回Nan错误，请问您这边处理了吗？

                else:
                    Q = alpha * self._S_object(pred, gt) + (1-alpha) * self._S_region(pred, gt)
                    if Q.item() < 0:
                        Q = torch.FloatTensor([0.0])
                if not isnan(Q.item()):
                    avg_q += Q.item()
                    img_num += 1.0
            avg_q /= img_num

我是这么处理的，不过与你们最近的EGnet工作发现有点出入，请问你们那边是怎么解决的？

Hanqer commented 5 years ago

@thb1314 您好，在计算Smeasure时，并没有遇到过Nan的情况（在几十个SOTA方法的结果上测试都没有问题）。能否提供详细的测试数据以及触发条件？

在S-measure的计算过程中，所有的除法操作都加上了极小值eps(1e-20)。我不是Smeasure的作者，所以我也不太清楚是否有其他情况会出现Nan。

此外，EGNet并未使用本代码进行评测。目前此代码的评测结果，与http://dpfan.net/d3netbenchmark/ 保持一致。

Thank you for your attention. Can you provide the triggering conditions of Nan, cause I haven't met the situation with Nan result(in over 30+ STOA methods).

In the processing of computing S-measure, all the division operation are legalized by adding a minimum value eps(1e-20). Since I am not the author of S-measure, I don't know if other conditions could cause the Nan result.

Besides, EGNet did not use this code for evaluation. The evaluation result of this code is consistent with http://dpfan.net/d3netbenchmark/ for now.

thb1314 commented 5 years ago

@Hanqer 您好，我在测试EGNet的PASCAL-S result map的时候会出现Nan的情况。 Thanks for your reply,I met this problem when I evaluate EGNet test results on the the PASCAL-S dataset.

thb1314 commented 5 years ago

@Hanqer BaiduYun Link Results (54gi)

Hanqer commented 5 years ago

@thb1314 嗨，我使用了你提供的Baidu Pan链接中的预测结果图。　测试了PASCAL-S数据集，其结果看VGG和Resnet50版本的均是正常的，通过本代码评测得到的S-measure分别为0.847和0.852，与原论文中的结果0.848和0.852基本一致，并没有出现你所遇到的Nan问题。 (刚刚新的commit更新了main中一个循环的顺序，但此更改并不影响评测的使用及其结果)

是否你的使用方法不太恰当?请参考Readme中所示的使用方法，如仍有问题，欢迎与我联系。 Email: hqer@foxmail.com WX: hanqer0123

1021151558 commented 4 years ago

也出现了nan...

Hanqer commented 4 years ago

@1021151558 Hi, after discussing with @thb1314, I find the nan error mainly because the GT images are not strictly binarized. I had fixed this bug, and if you still facing the nan error, please contact me and provide more details, so that I can reproduce this bug and fix it.

thb1314 commented 2 years ago

bug fixed

Hanqer / Evaluate-SOD

评估S-measure时候出现Nan错误 #1