search

TUI-NICR / ESANet

ESANet: Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis
Other
236 stars 51 forks source link

compute_class_weights issue for 894 labels #18

Open gnanukoth opened 3 years ago

gnanukoth commented 3 years ago

Hello, thank you so much for making the code publicly available. It is very helpful and easy to read and follow. I'm facing an issue with compute_class_weights function for 894 labels. For 13 and 40 labels it is working fine, but for 894 labels it show class weighting contains NaN. Not sure what might be the reason. Is it because of the classMapping.mat file? can you explain this file's role?

mona0809 commented 3 years ago

Maybe there are some categories which actually do not appear in the ground truth segmentation. This would result in zeros in the variable n_image_pixels_with_class and lead to NaN in division

gnanukoth commented 3 years ago

Thanks for getting back. I need a clarification regarding the depth image format for training and inference. For testing on custom procured RGB and depth images, what format should they be saved in? I tried saving the depth in mm as uint16 format. But the prediction results are not great, even after depth filling using colorisation technique. Is this approach right? Do you have any pointers for real time inference? May I know what depth map processing technique you followed in your robot system deployment experiment?

mona0809 commented 3 years ago

It depends on the dataset:

nyuv2: in mm, uint16 sundrgbd: in 1/10 mm, uint16 cityscapes: in m, float16

For application, we used sunrgbd model with images of a Kinect2. We did not use any depth inpainting methods. Just pay attention to use the parameter --raw_depth so that invalid depth values are set back to 0 after normalizing.