Test performance

[Ha-coding-user]

I train model like you and Train, Val Performance looks nice

But, Test score extracted by server is strange

========================== Arguments ==========================
dataset: /tmp/codalab/tmp7Q4hW1/run/input/ref predictions: /tmp/codalab/tmp7Q4hW1/run/input/res datacfg: /tmp/codalab/tmp7Q4hW1/run/program/semantic-kitti.yaml split: test output: /tmp/codalab/tmp7Q4hW1/run/output ===============================================================

[IOU EVAL] IGNORE: [] [IOU EVAL] INCLUDE: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19] Evaluating: 10% 20% 30% 40% 50% 60% 70% 80% 90% Done 🎉.

========================== RESULTS ==========================
Validation set: IoU avg 0.002 IoU class 1 [car] = 0.003 IoU class 2 [bicycle] = 0.000 IoU class 3 [motorcycle] = 0.000 IoU class 4 [truck] = 0.000 IoU class 5 [other-vehicle] = 0.000 IoU class 6 [person] = 0.000 IoU class 7 [bicyclist] = 0.000 IoU class 8 [motorcyclist] = 0.000 IoU class 9 [road] = 0.006 IoU class 10 [parking] = 0.001 IoU class 11 [sidewalk] = 0.006 IoU class 12 [other-ground] = 0.000 IoU class 13 [building] = 0.010 IoU class 14 [fence] = 0.001 IoU class 15 [vegetation] = 0.007 IoU class 16 [trunk] = 0.000 IoU class 17 [terrain] = 0.000 IoU class 18 [pole] = 0.000 IoU class 19 [traffic-sign] = 0.000 Precision = 6.16 Recall = 90.34 IoU Cmpltn = 6.12 mIoU SSC = 0.19

Is it true..?

[npurson]

Did you load the trained checkpoint to generate outputs?

We kept using the script to obtain the test score as reported. Since the train & val perforamce is right, I think you should check your test & submit process.

[npurson]

Assume it is solved since it haven't been updated for weeks