search

huang-yh / GaussianFormer

[ECCV 2024] Scene as Gaussians for Vision-Based 3D Semantic Occupancy Prediction
Other
262 stars 19 forks source link

EVAL time and results #32

Open kelo020304 opened 2 weeks ago

kelo020304 commented 2 weeks ago

How long will evaluation take? On my 4090 it takes more than 20minutes but it still not finish and I comment # load_from = 'ckpts/r101_dcn_fcos3d_pretrain.pth' in nuscenes_gs25600_solid.py ， and will these operation cause some bugs? image

LoickCh commented 1 week ago

You have ~6000 iters for 1 evaluation on nuscenes

kelo020304 commented 1 week ago

You have ~6000 iters for 1 evaluation on nuscenes

Thank you for your reply, its so fast

kelo020304 commented 1 week ago

You have ~6000 iters for 1 evaluation on nuscenes

10/14 16:44:32 - mmengine - INFO - tensor([ 1228626, 74016, 1153292, 7616559, 664560, 110560, 1049072, 221655, 888602, 3011142, 55378036, 2442130, 18847516, 25688440, 58281020, 61861948, 240793520], device='cuda:0', dtype=torch.int32) 10/14 16:44:32 - mmengine - INFO - tensor([ 2, 13, 6781, 5406, 10, 0, 1, 551, 4, 200, 77, 0, 98393, 184, 21288, 9014, 3983527], device='cuda:0', dtype=torch.int32) 10/14 16:44:32 - mmengine - INFO - tensor([ 18693, 907830, 27355038, 4062965, 83608, 3554, 6565, 7302669, 22779, 424973, 13621, 628, 34204064, 88684, 1832049, 432605, 76762576], device='cuda:0', dtype=torch.int32) 10/14 16:44:32 - mmengine - INFO - mIoU: 0.020197512613506774, iou2: 1.2703684568405151 This is my eval result, and I think these metrics is not correct , what problem here

LoickCh commented 1 week ago

If it is the official checkpoint, indeed they are not correct. You should have the one reported in the README.

Probably wrong evaluation command.

kelo020304 commented 1 week ago

If it is the official checkpoint, indeed they are not correct. You should have the one reported in the README

yes, I downloaded nuscenes_gs25600_solid from readme and do this : python eval.py --py-config config/nuscenes_gs25600_solid.py --work-dir out/nuscenes_gs25600_solid/ --resume-from out/nuscenes_gs25600_solid/state_dict.pth And then I get results mentioned above, and I'm confused if I need to comment : load_from = 'ckpts/r101_dcn_fcos3d_pretrain.pth'

But if I do not comment this , the evaluation can not run successfully