Questions about visible_mask

[aoyanl]

Hi, I see visible_mask in config is set False, so during training and evaluation, the voxels which is unobserved are calculated. Is my understanding correct？

[JeffWang987]

When visible_mask=Ture, we calculate the visible region (of camera/LiDAR), which is only used for visualization. Note that the visibility code is still under construction, and we do not use it in the training/validation phase

[aoyanl]

Will the visible mask be used in the training/validation phase later？

[JeffWang987]

We might leverage the mask later. In our early-stage experiment, we found that the mIoU of visible regions is much higher than that of invisible areas. And we encourage the community to investigate the influence of the visible mask (to boost performance or do something bigger).

[qihao-plus]

errors in @JeffWang987

bash tools/dist_test.sh ./projects/configs/baselines/CAM-R50_img1600_128x128x10.py \
    ./work_dirs/CAM-R50_img1600_128x128x10/best_SSC_mean_epoch_23.pth 8 \
    --show --show-dir ./work_dirs/CAM-R50_img1600_128x128x10/show

  File "/mnt/intel/jupyterhub/qihao/sync_projects/3_occupancy/OpenOccupancy/projects/occ_plugin/occupancy/apis/test.py", line 142, in custom_multi_gpu_test
    save_occ(result['pred_c'], result['pred_f'], data['img_metas'], out_dir, data['visible_mask'], data['gt_occ'])
KeyError: 'visible_mask'

extra: the model best_SSC_mean_epoch_23.pth is trained with CAM-R50_img1600_128x128x10.py visible_mask = False in default CAM-R50_img1600_128x128x10.py

[JeffWang987]

errors in @JeffWang987

bash tools/dist_test.sh ./projects/configs/baselines/CAM-R50_img1600_128x128x10.py \
    ./work_dirs/CAM-R50_img1600_128x128x10/best_SSC_mean_epoch_23.pth 8 \
    --show --show-dir ./work_dirs/CAM-R50_img1600_128x128x10/show

  File "/mnt/intel/jupyterhub/qihao/sync_projects/3_occupancy/OpenOccupancy/projects/occ_plugin/occupancy/apis/test.py", line 142, in custom_multi_gpu_test
    save_occ(result['pred_c'], result['pred_f'], data['img_metas'], out_dir, data['visible_mask'], data['gt_occ'])
KeyError: 'visible_mask'

extra: the model best_SSC_mean_epoch_23.pth is trained with CAM-R50_img1600_128x128x10.py visible_mask = False in default CAM-R50_img1600_128x128x10.py

Yes, we haven't released the training code with the mask. For now, it's only used for visualization.

[qihao-plus]

so I have to wait your code with mask for visualization ? Thanks!

[qihao-plus]

Also, how to visualize GT voxel in OpenOccupancy/data/nuScenes-Occupancy/v0.0-base/scene_aa45d94630be481787f20121ce89a605/occupancy/548ca66e9adc48ca88d1bcaeeafc35da.npy Thanks!

shape: (136774, 4)

[JeffWang987]

Also, how to visualize GT voxel in OpenOccupancy/data/nuScenes-Occupancy/v0.0-base/scene_aa45d94630be481787f20121ce89a605/occupancy/548ca66e9adc48ca88d1bcaeeafc35da.npy Thanks!

shape: (136774, 4)

The Visualization code is being reframed (might be released in the future), you can DIY yours by referring to MonoScene

[onionysy]

Also, how to visualize GT voxel in OpenOccupancy/data/nuScenes-Occupancy/v0.0-base/scene_aa45d94630be481787f20121ce89a605/occupancy/548ca66e9adc48ca88d1bcaeeafc35da.npy Thanks! shape: (136774, 4)

The Visualization code is being reframed (might be released in the future), you can DIY yours by referring to MonoScene

I see that you have a Visualization of your network in the "Visualization of CONet" section of Readme, how did you visualize it? Or do you have the code, but not yet open source?

[onionysy]

Also, how to visualize GT voxel in OpenOccupancy/data/nuScenes-Occupancy/v0.0-base/scene_aa45d94630be481787f20121ce89a605/occupancy/548ca66e9adc48ca88d1bcaeeafc35da.npy Thanks! shape: (136774, 4)

The Visualization code is being reframed (might be released in the future), you can DIY yours by referring to MonoScene

We found that the current MoneScene visualization process uses pkl files, but the current openocc visualization process saves npy files, which is not the same. How can npy files be visualized?

[Qinyu-Z]

We have the same question with this.

[jackwky]

中的错误

bash tools/dist_test.sh ./projects/configs/baselines/CAM-R50_img1600_128x128x10.py \
    ./work_dirs/CAM-R50_img1600_128x128x10/best_SSC_mean_epoch_23.pth 8 \
    --show --show-dir ./work_dirs/CAM-R50_img1600_128x128x10/show

  File "/mnt/intel/jupyterhub/qihao/sync_projects/3_occupancy/OpenOccupancy/projects/occ_plugin/occupancy/apis/test.py", line 142, in custom_multi_gpu_test
    save_occ(result['pred_c'], result['pred_f'], data['img_metas'], out_dir, data['visible_mask'], data['gt_occ'])
KeyError: 'visible_mask'

额外：模型默认使用best_SSC_mean_epoch_23.pth``CAM-R50_img1600_128x128x10.py``visible_mask = False``CAM-R50_img1600_128x128x10.py

我想问一下你这个路径./work_dirs/CAM-R50_img1600_128x128x10/show 是通过什么操作生成的？是你自己训练过的吗？