How PasCo addresses the invalid masks from SSCBench-KITTI360

[haiphamcse]

Hi there, loved your work. I was wondering when you are using SSCBench-KITTI360, how did you handle the invalid mask for PasCo? From my experience, when using this mask in training, the model will tend to generate weird artifacts for the sky regions (here I have an example from MonoScene). I would love to hear how you handle this and whether PasCo has these artifacts?

[anhquancao]

Hello @haiphamcse, I simply increased the weight of the empty class during training. However, this issue is less severe in Pasco, as it utilizes a point cloud as input, which is easier than using an image.

[haiphamcse]

So from my understanding, you still use these invalid masks for PasCo while training, however you increased the weight of empty classes to mitigate this? I still think that as these invalid masks covers the sky of mosts scenes as invalids, the model should not be able to learn that the sky is empty. It is surprising to me how increasing the weight of empty class solves this

[anhquancao]

In my situation, I use LiDAR point clouds as input, which makes this issue less problematic. Therefore, I guess adjusting the weights is enough for my needs. Maybe a simple solution is to randomly set e.g. xx% of unknown voxels as empty space during training.