search

Haiyang-W / DSVT

[CVPR2023] Official Implementation of "DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets"
https://arxiv.org/abs/2301.06051
Apache License 2.0
373 stars 28 forks source link

Question TensorRT deployment & PCDET #60

Closed d33dler closed 1 year ago

d33dler commented 1 year ago

Great work, and nice that you help the user-base ! I have a question regarding full RT deployment. I trained the model using PCDET config. And this user went the extra mile to implement the plugins for all model parts. Would it be possible to deploy using his work? Or do I have to implement additional plugins?

d33dler commented 1 year ago

Update: I researched a bit. Firstly: Currently your ONNX conversion has an error in AllDSVTBlocksTRT : inputs = (outputs, set_voxel_inds, set_voxel_masks, pos_embed, True) outputs = self.dsvtblocks_list[blc_id].encoder_list[set_id](*inputs) The boolean shouldn't be there and causes TypeError: forward() takes from 4 to 5 positional arguments but 6 were given. I don't understand how this wasn't causing issues. Secondly: from this deployment script it seems you're converting to ONNX only the DSVT blocks. I assume if I want to deploy the whole model I need to adapt the code?

chenshi3 commented 1 year ago

Update: I researched a bit. Firstly: Currently your ONNX conversion has an error in AllDSVTBlocksTRT : inputs = (outputs, set_voxel_inds, set_voxel_masks, pos_embed, True) outputs = self.dsvtblocks_list[blc_id].encoder_list[set_id](*inputs) The boolean shouldn't be there and causes TypeError: forward() takes from 4 to 5 positional arguments but 6 were given. I don't understand how this wasn't causing issues. Secondly: from this deployment script it seems you're converting to ONNX only the DSVT blocks. I assume if I want to deploy the whole model I need to adapt the code?

For the question one, the boolean variable is used in onnx_export. For the sake of code simplicity, we only provide the deployment code for backbone3d. The deployment of other parts is straightforward.

chenshi3 commented 1 year ago

Great work, and nice that you help the user-base ! I have a question regarding full RT deployment. I trained the model using PCDET config. And this user went the extra mile to implement the plugins for all model parts. Would it be possible to deploy using his work? Or do I have to implement additional plugins?

We do not test the deployment code provided by this.

d33dler commented 1 year ago

Update: I researched a bit. Firstly: Currently your ONNX conversion has an error in AllDSVTBlocksTRT : inputs = (outputs, set_voxel_inds, set_voxel_masks, pos_embed, True) outputs = self.dsvtblocks_list[blc_id].encoder_list[set_id](*inputs) The boolean shouldn't be there and causes TypeError: forward() takes from 4 to 5 positional arguments but 6 were given. I don't understand how this wasn't causing issues. Secondly: from this deployment script it seems you're converting to ONNX only the DSVT blocks. I assume if I want to deploy the whole model I need to adapt the code?

For the question one, the boolean variable is used in onnx_export. For the sake of code simplicity, we only provide the deployment code for backbone3d. The deployment of other parts is straightforward.

i don't have experience with ONNX conversion. is it possible to provide a minimal example of conversion of another part of this model ?

Mavericky-j commented 9 months ago

Update: I researched a bit. Firstly: Currently your ONNX conversion has an error in AllDSVTBlocksTRT : inputs = (outputs, set_voxel_inds, set_voxel_masks, pos_embed, True) outputs = self.dsvtblocks_list[blc_id].encoder_list[set_id](*inputs) The boolean shouldn't be there and causes TypeError: forward() takes from 4 to 5 positional arguments but 6 were given. I don't understand how this wasn't causing issues. Secondly: from this deployment script it seems you're converting to ONNX only the DSVT blocks. I assume if I want to deploy the whole model I need to adapt the code?

you should reinstall pcdet of DSVT rather than using UniTR's pcdet