请问能否开源计算RMSE的代码？参考PENET参考计算你开源的GNS模型的RMSE和论文中差距太大？

[fanrz]

have you reproduced the similar result in GuideNet paper?

[neverstoplearn]

have you reproduced the similar result in GuideNet paper?

not yet

[yudmoe]

The code calculate RMSE is just locate in criteria.py, but I am not sure did it be commit when you make this issue? Here is some RMSE of my test in the depth_selection annotated 1K val dataset。 the RMSE of given GN model is 0.7640976475179195(m) the RMSE of given GNS model is 0.7579560613930225(m)

the RMSE of my trained model is 0.6275460012257099(m)

This outcome of my trained model surprise me a lot, but since the training use the train (about 86000) and val set (about 6000) of data_depth_annotated, and the test use some flip trick (in the test function) , and the test img size is 256*1216 but not the full image, maybe it make some sense, also show the potential of this model

[fanrz]

The code calculate RMSE is just locate in criteria.py, but I am not sure did it be commit when you make this issue? Here is some RMSE of my test in the depth_selection annotated 1K val dataset。 the RMSE of given GN model is 0.7640976475179195(m) the RMSE of given GNS model is 0.7579560613930225(m)

the RMSE of my trained model is 0.6275460012257099(m)

This outcome of my trained model surprise me a lot, but since the training use the train (about 86000) and val set (about 6000) of data_depth_annotated, and the test use some flip trick (in the test function) , and the test img size is 256*1216 but not the full image, maybe it make some sense, also show the potential of this model

Hi, @yudmoe FULL size 1216*352 is must in the val and test set. or you can submit your result in KITTI benchmark to get score. In my idea, 0.62754 may be wrong.

[yudmoe]

OK， I will amend my code and then test in the resolution of origin image，thank you for reply

[yudmoe]

Got my mistake, actually,I have made two mistake: (1)didn't use the FULL size 1216*352, when I test in the full size, I got RMSE 0.79889674 in the depth_selection 1K val dataset, (2) depth_selection 1K val dataset is a subset of the val set of data_depth_annotated (about 6000), which mean I use the training dataset to test. For the KITTI benchmark, I need to apply for submission first, but the KITTI benchmark test outcome will be worse than 0.798 because upper reason. By the way, what result did you get?

[fanrz]

Got my mistake, actually,I have made two mistake: (1)didn't use the FULL size 1216*352, when I test in the full size, I got RMSE 0.79889674 in the depth_selection 1K val dataset, (2) depth_selection 1K val dataset is a subset of the val set of data_depth_annotated (about 6000), which mean I use the training dataset to test. For the KITTI benchmark, I need to apply for submission first, but the KITTI benchmark test outcome will be worse than 0.798 because upper reason. By the way, what result did you get?

I can not remember the number, but it is a much worse results than ones in the paper. I do not know whether anyone else reproduce the experiments.

[yudmoe]

Got my mistake, actually,I have made two mistake: (1)didn't use the FULL size 1216*352, when I test in the full size, I got RMSE 0.79889674 in the depth_selection 1K val dataset, (2) depth_selection 1K val dataset is a subset of the val set of data_depth_annotated (about 6000), which mean I use the training dataset to test. For the KITTI benchmark, I need to apply for submission first, but the KITTI benchmark test outcome will be worse than 0.798 because upper reason. By the way, what result did you get?

I can not remember the number, but it is a much worse results than ones in the paper. I do not know whether anyone else reproduce the experiments.

I have made some error visualization work of this GuideNet, maybe the training stratege that fix the training image to the bottom 256 pixels part of dataset is the key of not making good result, lot of dist predict error are intensively distributie in the upper side of test image

[fanrz]

the bottom crop 256 pixels is ok in KITTI depth completion task, cause few degradation. I have tested this stratege.

[yudmoe]

the bottom crop 256 pixels is ok in KITTI depth completion task, cause few degradation. I have tested this stratege.

thank you for notice me