search

JeffWang987 / OpenOccupancy

[ICCV 2023] OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception
Apache License 2.0
592 stars 50 forks source link

Questions about visible_mask #13

Open aoyanl opened 1 year ago

aoyanl commented 1 year ago

Hi, I see visible_mask in config is set False, so during training and evaluation, the voxels which is unobserved are calculated. Is my understanding correct？

JeffWang987 commented 1 year ago

When visible_mask=Ture, we calculate the visible region (of camera/LiDAR), which is only used for visualization. Note that the visibility code is still under construction, and we do not use it in the training/validation phase

aoyanl commented 1 year ago

Will the visible mask be used in the training/validation phase later？

JeffWang987 commented 1 year ago

We might leverage the mask later. In our early-stage experiment, we found that the mIoU of visible regions is much higher than that of invisible areas. And we encourage the community to investigate the influence of the visible mask (to boost performance or do something bigger).

qihao-plus commented 1 year ago

errors in @JeffWang987 

bash tools/dist_test.sh ./projects/configs/baselines/CAM-R50_img1600_128x128x10.py \
    ./work_dirs/CAM-R50_img1600_128x128x10/best_SSC_mean_epoch_23.pth 8 \
    --show --show-dir ./work_dirs/CAM-R50_img1600_128x128x10/show

  File "/mnt/intel/jupyterhub/qihao/sync_projects/3_occupancy/OpenOccupancy/projects/occ_plugin/occupancy/apis/test.py", line 142, in custom_multi_gpu_test
    save_occ(result['pred_c'], result['pred_f'], data['img_metas'], out_dir, data['visible_mask'], data['gt_occ'])
KeyError: 'visible_mask'

extra: the model best_SSC_mean_epoch_23.pth is trained with CAM-R50_img1600_128x128x10.py visible_mask = False in default CAM-R50_img1600_128x128x10.py

JeffWang987 commented 1 year ago

errors in @JeffWang987

bash tools/dist_test.sh ./projects/configs/baselines/CAM-R50_img1600_128x128x10.py \
    ./work_dirs/CAM-R50_img1600_128x128x10/best_SSC_mean_epoch_23.pth 8 \
    --show --show-dir ./work_dirs/CAM-R50_img1600_128x128x10/show
  File "/mnt/intel/jupyterhub/qihao/sync_projects/3_occupancy/OpenOccupancy/projects/occ_plugin/occupancy/apis/test.py", line 142, in custom_multi_gpu_test
    save_occ(result['pred_c'], result['pred_f'], data['img_metas'], out_dir, data['visible_mask'], data['gt_occ'])
KeyError: 'visible_mask'

extra: the model best_SSC_mean_epoch_23.pth is trained with CAM-R50_img1600_128x128x10.py visible_mask = False in default CAM-R50_img1600_128x128x10.py

Yes, we haven't released the training code with the mask. For now, it's only used for visualization.

qihao-plus commented 1 year ago

so I have to wait your code with mask for visualization ? Thanks!

qihao-plus commented 1 year ago

Also, how to visualize GT voxel in OpenOccupancy/data/nuScenes-Occupancy/v0.0-base/scene_aa45d94630be481787f20121ce89a605/occupancy/548ca66e9adc48ca88d1bcaeeafc35da.npy Thanks!

shape: (136774, 4)

JeffWang987 commented 1 year ago

Also, how to visualize GT voxel in OpenOccupancy/data/nuScenes-Occupancy/v0.0-base/scene_aa45d94630be481787f20121ce89a605/occupancy/548ca66e9adc48ca88d1bcaeeafc35da.npy Thanks!

shape: (136774, 4)

The Visualization code is being reframed (might be released in the future), you can DIY yours by referring to MonoScene

onionysy commented 1 year ago

Also, how to visualize GT voxel in OpenOccupancy/data/nuScenes-Occupancy/v0.0-base/scene_aa45d94630be481787f20121ce89a605/occupancy/548ca66e9adc48ca88d1bcaeeafc35da.npy Thanks! shape: (136774, 4)

The Visualization code is being reframed (might be released in the future), you can DIY yours by referring to MonoScene

I see that you have a Visualization of your network in the "Visualization of CONet" section of Readme, how did you visualize it? Or do you have the code, but not yet open source?

onionysy commented 1 year ago

Also, how to visualize GT voxel in OpenOccupancy/data/nuScenes-Occupancy/v0.0-base/scene_aa45d94630be481787f20121ce89a605/occupancy/548ca66e9adc48ca88d1bcaeeafc35da.npy Thanks! shape: (136774, 4)

The Visualization code is being reframed (might be released in the future), you can DIY yours by referring to MonoScene

We found that the current MoneScene visualization process uses pkl files, but the current openocc visualization process saves npy files, which is not the same. How can npy files be visualized?

Qinyu-Z commented 1 year ago

We have the same question with this.

jackwky commented 10 months ago

中的错误

bash tools/dist_test.sh ./projects/configs/baselines/CAM-R50_img1600_128x128x10.py \
    ./work_dirs/CAM-R50_img1600_128x128x10/best_SSC_mean_epoch_23.pth 8 \
    --show --show-dir ./work_dirs/CAM-R50_img1600_128x128x10/show
  File "/mnt/intel/jupyterhub/qihao/sync_projects/3_occupancy/OpenOccupancy/projects/occ_plugin/occupancy/apis/test.py", line 142, in custom_multi_gpu_test
    save_occ(result['pred_c'], result['pred_f'], data['img_metas'], out_dir, data['visible_mask'], data['gt_occ'])
KeyError: 'visible_mask'

额外：模型默认使用best_SSC_mean_epoch_23.pth``CAM-R50_img1600_128x128x10.py``visible_mask = False``CAM-R50_img1600_128x128x10.py

我想问一下你这个路径./work_dirs/CAM-R50_img1600_128x128x10/show 是通过什么操作生成的？是你自己训练过的吗？

YuxiChen-GL commented 3 months ago

So if we can visualize the result through this open-sourced code now？

YuxiChen-GL commented 3 months ago

Also, how to visualize GT voxel in OpenOccupancy/data/nuScenes-Occupancy/v0.0-base/scene_aa45d94630be481787f20121ce89a605/occupancy/548ca66e9adc48ca88d1bcaeeafc35da.npy Thanks! shape: (136774, 4)

The Visualization code is being reframed (might be released in the future), you can DIY yours by referring to MonoScene

I see that you have a Visualization of your network in the "Visualization of CONet" section of Readme, how did you visualize it? Or do you have the code, but not yet open source?

Did you visualize it successfully？