Lidar segmentation task class 0 is not allowed submission failure issue

[engomarwasfy]

Greetings, Authors

Thank you for your great work.

I have a question regarding validation on the submission file for the Lidar segmentation task https://github.com/nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/eval/lidarseg/validate_submission.py#:~:text=import%20argparse%0Aimport,160 the validation doesn't allow class 0 to be in predictions and this same error I face at Eval AI.

I believe this is unclear why you don't accept class 0 although the description of the task here https://www.nuscenes.org/lidar-segmentation?externalData=all&mapData=all&modalities=Any#:~:text=Classes-,The%20nuScenes-lidarseg%20dataset,vehicle.ego,-1 it is mentioned that class 0 is one of the predictions.

I believe that evaluation should accept predictions with classes [0,16] and evaluate on classes [1,16] only and ignore class 0 points but should not reject the submission because of existing class 0 predictions.

please your support is needed and in case class 0 is not allowed what should be its mapped class from [1,16]

[whyekit-motional]

@engomarwasfy class 0 is the void / ignore class (as shown in your link / here).

The nuScenes lidar semantic challenge is one which involves 16 classes, so usually users would train a model with 16 classes, rather than a model with 17 classes (16 classes + 1 ignore class). Hence, we help users to check that their submissions do not contain 0 here.

If you have trained a model using 17 classes, you could map all 0s in your submission to some arbitrary number, since we drop all predictions that correspond to a ground truth of 0 here.

[engomarwasfy]

Great thank you for clarification @whyekit-motional.