viewpoint.json

[FengLoveBella]

Thanks for the code, nice work. Would you share the 'viewpoint.json' file which used in 'show_results_scannet.py'? When I tried to run the code I found the code cannot find the 'viewpoint.json'.

[SumingNTUT]

I have same problem, if you know how to fix it,

please tell me

Thank you

[FengLoveBella]

@SumingNTUT By the way, I found when I use the code, I got a plausible mAP, however, when I use show_result_scannet to visualize the result, I found the results are bad, although the BBs are okay, however the classification results are almost wrong.

[zaiweizhang]

@GitBoSun Would you mind looking at this issue?

[GitBoSun]

Hi, thanks for your interest to our work! I have uploaded viewpoint.json to the utils folder. As for the object color, there is color_mapping for each semantic label in line29. We assume the last dimension of the ouput boundingboxes denotes the semantic labels (line 108-111). Did you pass the correct semantic labels?

[FengLoveBella]

@zaiweizhang @GitBoSun Thanks for your reply. I am afraid 'color mapping' which used in the code is not non-proper (I used the original code directly). I found a very interesting thing, I used the code to train a model (360 epoch), and the mAP@ 0.25 = eval mAP: 0.660502, and mAP@0.5 = eval mAP: 0.472760, I suppose which means the bounding box prediction is okay. However, when I try to visualize the prediction result by 'show_results_scannet.py', I got a very poor results (bounding box is okay, but the label is almost wrong). I tried to debug the code, I found for the first test file(scene0011_00), which the bouding box label is: [ 5, 39, 39, 5, 5, 5, 5, 7, 7, 5, 5, 8, 8, 5, 5, 5, 5, 24, 24, 3, 3, 12, 34, 3, 9, 9] However the prediction bouding box label is: [33, 33, 33, 33, 33, 33, 33, 33, 33, 28, 33, 11, 33, 33, 33, 11, 33, 33, 10, 10, 10, 33, 10, 33, 10, 36, 33, 33, 28, 33, 10, 10, 33, 33, 11, 33, 33] the visualization is as follows:

[GitBoSun]

This looks weird for me. Could you please provide the MAP for each category in the training log?

[FengLoveBella]

@GitBoSun the log_train file is as follows: log_train.txt

[FengLoveBella]

@GitBoSun Sure.

eval cabinet Average Precision: 0.454509 eval bed Average Precision: 0.899022 eval chair Average Precision: 0.927298 eval sofa Average Precision: 0.910974 eval table Average Precision: 0.636143 eval door Average Precision: 0.563234 eval window Average Precision: 0.533729 eval bookshelf Average Precision: 0.506288 eval picture Average Precision: 0.181430 eval counter Average Precision: 0.545911 eval desk Average Precision: 0.749221 eval curtain Average Precision: 0.617094 eval refrigerator Average Precision: 0.583707 eval showercurtrain Average Precision: 0.687724 eval toilet Average Precision: 0.981283 eval sink Average Precision: 0.672231 eval bathtub Average Precision: 0.910117 eval garbagebin Average Precision: 0.529130 eval mAP: 0.660502 eval cabinet Recall: 0.817204 eval bed Recall: 0.950617 eval chair Recall: 0.956871 eval sofa Recall: 0.989691 eval table Recall: 0.860000 eval door Recall: 0.758030 eval window Recall: 0.755319 eval bookshelf Recall: 0.896104 eval picture Recall: 0.333333 eval counter Recall: 0.884615 eval desk Recall: 0.968504 eval curtain Recall: 0.805970 eval refrigerator Recall: 0.947368 eval showercurtrain Recall: 0.928571 eval toilet Recall: 1.000000 eval sink Recall: 0.785714 eval bathtub Recall: 0.967742 eval garbagebin Recall: 0.801887 eval AR: 0.855975 eval cabinet Average Precision: 0.236958 eval bed Average Precision: 0.837149 eval chair Average Precision: 0.812462 eval sofa Average Precision: 0.813043 eval table Average Precision: 0.520738 eval door Average Precision: 0.258810 eval window Average Precision: 0.202301 eval bookshelf Average Precision: 0.399841 eval picture Average Precision: 0.047947 eval counter Average Precision: 0.364429 eval desk Average Precision: 0.495830 eval curtain Average Precision: 0.288524 eval refrigerator Average Precision: 0.392110 eval showercurtrain Average Precision: 0.388540 eval toilet Average Precision: 0.900329 eval sink Average Precision: 0.376733 eval bathtub Average Precision: 0.808517 eval garbagebin Average Precision: 0.365413 eval mAP: 0.472760 eval cabinet Recall: 0.556452 eval bed Recall: 0.888889 eval chair Recall: 0.866228 eval sofa Recall: 0.938144 eval table Recall: 0.751429 eval door Recall: 0.481799 eval window Recall: 0.404255 eval bookshelf Recall: 0.727273 eval picture Recall: 0.112613 eval counter Recall: 0.615385 eval desk Recall: 0.834646 eval curtain Recall: 0.462687 eval refrigerator Recall: 0.754386 eval showercurtrain Recall: 0.607143 eval toilet Recall: 0.913793 eval sink Recall: 0.510204 eval bathtub Recall: 0.838710 eval garbagebin Recall: 0.601887 eval AR: 0.659218

[yanghtr]

@zhoufengbuaa If you use the provided pretrained checkpoint, can you get reasonable bounding box predictions? If the mAP is reasonable, the predicted label should also be reasonable. If the network is trained correctly, it should predict reasonable semantic.

[FengLoveBella]

@yanghtr Thanks for the reply. The mAP is plausible, however, the classification label is horrible, which is really weird. I cannot find the pre-trained model, could you share it which we can use directly. There would be very useful for testing there is something wrong with my training process or something else. I did find there is a pre-trained model (https://github.com/facebookresearch/DepthContrast), however, in my opinion, it is not a pre-trained model of your model.

[zaiweizhang]

I do not think the prediction results are awfully off. Since the code has been tested through multiple third party users, I think the color mapping or the label mapping is wrong. Would you mind providing some insights? @GitBoSun

[yanghtr]

@zhoufengbuaa There is a pretrained model here: https://github.com/zaiweizhang/H3DNet/issues/11

[GitBoSun]

@zaiweizhang Their GT colors are correct and predicted semantic labels are wrong. Since the output labels are [33, 33, 33, 33, 33, 33, 33, 33, 33, 28, 33, 11, 33, 33, 33, 11, 33, 33, 10, 10, 10, 33, 10, 33, 10, 36, 33, 33, 28, 33, 10, 10, 33, 33, 11, 33, 33], they had the colors corresponding to 33. I'm not sure what happened.

[FengLoveBella]

@yanghtr @zaiweizhang I used the pre-trained model and got a similar results. . And the log_eval file is as follows log_eval.txt

[zaiweizhang]

Hi,

The issue is the dump helper fuction. We did not dump the correct prediction labels. Now we have fixed the issue. Please check! Thanks for the help in solving this issue! We really appreciate it.

Zaiwei

[FengLoveBella]

@zaiweizhang Cool, the problem is solved, come again, good job.

[Dhanalaxmi17]

Hi @zaiweizhang and @GitBoSun, Thank you for this repo. I am trying to see the visualizations, but when I ran show_results_scannet.py, I could only get the bounding boxes, but how do I get the scene and bounding boxes on top of it. can you please help me to get this issue solved for me. Thank you