search

Haiyang-W / DSVT

[CVPR2023] Official Implementation of "DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets"
https://arxiv.org/abs/2301.06051
Apache License 2.0
373 stars 28 forks source link

Setup for Nuscenes #22

Closed Flamyx closed 1 year ago

Flamyx commented 1 year ago

Hello! I have some issues with configuring DSVT-P for Nuscenes. In particular, i took POINT_CLOUD_RANGE and VOXEL_SIZE from nuscenes_dataset.yaml with dsvt_plain_1f_onestage.yaml, but model forward fails in PointPillarScatter3d due to indices being out of range. Could you please provide configuration for DSVT models on Nuscenes or the logic on how to choose POINT_CLOUD_RANGE and VOXEL_SIZE with respect to model configurations?

Haiyang-W commented 1 year ago

Hi, sorry for the late reply. The point cloud range we used is "POINT_CLOUD_RANGE: [-54.0, -54.0, -5.0, 54.0, 54.0, 3.0]". The voxel size is "VOXEL_SIZE: [0.3, 0.3, 8.0]". The sparse shape of DSVT is "sparse_shape: [360, 360, 1]". Note that 54*2/0.3 = 360.

As for the configures of DSVT in NuScenes, @chenshi3 is working on compatibility with transfusion on the official PCDet code base, we will release configure of nuscenes once it is compatible. Please wait patiently, all the questions can be raised in the issue, and we will reply in time.

Haiyang

Haiyang-W commented 1 year ago

Hello! I have some issues with configuring DSVT-P for Nuscenes. In particular, i took POINT_CLOUD_RANGE and VOXEL_SIZE from nuscenes_dataset.yaml with dsvt_plain_1f_onestage.yaml, but model forward fails in PointPillarScatter3d due to indices being out of range. Could you please provide configuration for DSVT models on Nuscenes or the logic on how to choose POINT_CLOUD_RANGE and VOXEL_SIZE with respect to model configurations?

Have you implemented it successfully? We will release the nuscenes code in a few weeks.

Moreover, we have merged Transfusion-L into OpenPCDet, please see pcdet-transfusion. I think with the current dsvt repo and official PCDet, you can easily reproduce nuscenes' results.

Please note the difference of nuscenes and waymo

Anyone who needs nuscenes urgently is welcome to email me and I can share an internal version in advance.

Flamyx commented 1 year ago

Hello, thank you for your answers! I've addopted DSVT-Pillar version from waymo with adjusted hyperparameters for nuscenes. I've tried Transfusion but model forward breaks due to HeightCompression output shape incompatibility with DSVT required input shape. I'll wait for your nuscenes code release to see if there is any other way to improve the model.

chenshi3 commented 1 year ago

Hello! I am currently cleaning the code for DSVT on nuscenes. Due to other tasks keeping me busy, I anticipate completing this within the next one to two weeks.

Haiyang-W commented 1 year ago

Hello, thank you for your answers! I've addopted DSVT-Pillar version from waymo with adjusted hyperparameters for nuscenes. I've tried Transfusion but model forward breaks due to HeightCompression output shape incompatibility with DSVT required input shape. I'll wait for your nuscenes code release to see if there is any other way to improve the model.

code of nuScenes is released.