hustvl / Symphonies

[CVPR 2024] Symphonies (Scene-from-Insts): Symphonize 3D Semantic Scene Completion with Contextual Instance Queries
https://arxiv.org/abs/2306.15670
MIT License
168 stars 6 forks source link

Voxel Proposal Layer for KITTI-360 #9

Closed haiphamcse closed 6 months ago

haiphamcse commented 9 months ago

Hi there, loved your work. I want to ask whether your results on KITTI-360 uses the voxel proposal layer (I do see in the log that it does) and which model did you use for the depth prediction?

npurson commented 9 months ago
  1. Yes, our implementation for KITTI-360 indeed incorporates the VPL. The overall network architecture remains consistent across both datasets.
  2. Regarding the depth prediction model, we employ the same pre-trained MobileStereoNet for both the KITTI-360 and SemanticKITTI datasets. This decision is based on the considerable similarities in image characteristics among the KITTI2015, SemanticKITTI, and KITTI-360 datasets.

We appreciate your highlighting the need for clarity regarding depth generation in our documentation. We are in the process of revising it to address this. In the meantime, you may proceed by following the instructions provided for SemanticKITTI, as the process for depth generation is analogous.

haiphamcse commented 9 months ago

Thank you for the quick reply, can you provide the code to export (or the preprocessed pseudo point clouds) of MobileStereoNet on KITTI-360?

npurson commented 9 months ago

Regrettably, we are currently unable to share the code or preprocessed results. The raw stereo image data and associated code environment have been cleaned, and the size of proprocessed depth data exceeds ~80 GB, making it impractical for upload.

However, we would like to guide you on implementing KITTI-360 preprocessing using the existing MobileStereoNet codebase from VoxFormer. You can achieve this by making only the following two key modifications:

  1. Adapting of disparity value.
  2. Matching stereo frames between SSCBench and original KITTI-360. You can directly utilize the matched results.