Haiyang-W / DSVT

[CVPR2023] Official Implementation of "DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets"
https://arxiv.org/abs/2301.06051
Apache License 2.0
372 stars 28 forks source link

TensorRT Deployment for nuScenes Trained Model #46

Closed yusufy1ld1z closed 1 year ago

yusufy1ld1z commented 1 year ago

Hi, thanks for a great job! I am having trouble converting the weights of the model trained with nuScenes dataset. As I understand, the deploy.py file only works with Waymo compatibility, and the nuScenes trtengine config yaml is not released. I have tried to write the nuScenes trtengine config file by just replacing the BACKBONE_3D part of the yaml, but before coming to this stage I had other problems with deploy.py. The input data (batch_dict) given by you is Waymo data and I think it is not a solution to just change this data with nuScenes because input_shapes and dynamic_axes config must be changed for nuScenes compatibility. So, is there a way to convert models trained with nuScenes to TensorRT right now, can I resolve this by myself? Are you planning to release some dataset-diagnostic deploy code in the near future?

Haiyang-W commented 1 year ago

Hi, thanks for a great job! I am having trouble converting the weights of the model trained with nuScenes dataset. As I understand, the deploy.py file only works with Waymo compatibility, and the nuScenes trtengine config yaml is not released. I have tried to write the nuScenes trtengine config file by just replacing the BACKBONE_3D part of the yaml, but before coming to this stage I had other problems with deploy.py. The input data (batch_dict) given by you is Waymo data and I think it is not a solution to just change this data with nuScenes because input_shapes and dynamic_axes config must be changed for nuScenes compatibility. So, is there a way to convert models trained with nuScenes to TensorRT right now, can I resolve this by myself? Are you planning to release some dataset-diagnostic deploy code in the near future?

Thanks for your interesting in our work!

Haiyang-W commented 1 year ago

NuScenes Deployment of DSVT will be released in our new paper (Unified multi-modal DSVT), which is accepted by ICCV2023. Please be patient.