search

IDEA-Research / 3D-deformable-attention

[ICCV 2023] Official implementation of the paper "DFA3D: 3D Deformable Attention For 2D-to-3D Feature Lifting"
Other
150 stars 2 forks source link

Can DFA3D module be exported to TensorRT? #5

Closed QingXIA233 closed 1 year ago

QingXIA233 commented 1 year ago

Hello, really enlightening works! I wonder whether it's possible to convert your DFA3D to onnx and TensorRT models. Currently, TensorRT has supported MultiScaleDeformableAttention 2D, did you consider the deployment of DFA3D or should I write TensorRT plugin for it by myself?

QingXIA233 commented 1 year ago

BTW, If I click the config link of DFA3D-based BevFormers in the model zoo in README page: image I got this error: image Perhaps there're some errors?

LHY-HongyangLi commented 1 year ago

Hello, really enlightening works! I wonder whether it's possible to convert your DFA3D to onnx and TensorRT models. Currently, TensorRT has supported MultiScaleDeformableAttention 2D, did you consider the deployment of DFA3D or should I write TensorRT plugin for it by myself?

Hi @QingXIA233 , I'm sorry for my late reply. Sure, we would like to implement DFA3D to onnx and TensorRT to convenient the users in AutoDriving community. However, recently I'm busy with something others, and can not pay attention to the implementation. We welcome the contribution of the community users.

LHY-HongyangLi commented 1 year ago

BTW, If I click the config link of DFA3D-based BevFormers in the model zoo in README page: image I got this error: image Perhaps there're some errors?

I'm sorry for our mistake 😂. We have fixed this bug, thank you very much!

QingXIA233 commented 1 year ago

Hello, really enlightening works! I wonder whether it's possible to convert your DFA3D to onnx and TensorRT models. Currently, TensorRT has supported MultiScaleDeformableAttention 2D, did you consider the deployment of DFA3D or should I write TensorRT plugin for it by myself?

Hi @QingXIA233 , I'm sorry for my late reply. Sure, we would like to implement DFA3D to onnx and TensorRT to convenient the users in AutoDriving community. However, recently I'm busy with something others, and can not pay attention to the implementation. We welcome the contribution of the community users.

@LHY-HongyangLi Thank you for the reply. I understand your position. I'm currently trying to convert DFA3D to onnx and trt. BTW, I see that you have the plan of releasing "Format and release the code of "preparing depth map"", may I ask when this will be completed, or could you please give me some hints of generation depth map gt? Currently, I generate my depth gt by using lidar points, intrinsic parameters and mei projection, I also set depth_map[depth_map > 60] = 0. The shape is (B, N, H, W), where N stands for the camera num, (H, W) is the height and width of the input images. I wonder whether you performed other processes on depth map gt or what I did is ok. Thanks advance for your help.

Best regards

LHY-HongyangLi commented 1 year ago

Hi @QingXIA233 , I'm grateful for your attention and welcome your contribution! I'm recently busy with something others, so "Format and release the code of "preparing depth map" may be delayed until next month. You can refer to SOLOFusion or BEVDepth for the generating of depth map. Good luck!

LHY-HongyangLi commented 1 year ago

I'll close this issue, if you have any further questions, feel free to reopen it.