search

NVIDIA-AI-IOT / Lidar_AI_Solution

A project demonstrating Lidar related AI solutions, including three GPU accelerated Lidar/camera DL networks (PointPillars, CenterPoint, BEVFusion) and the related libs (cuPCL, 3D SparseConvolution, YUV2RGB, cuOSD,).
Other
1.23k stars 217 forks source link

Does 3D backbone support PTQ #119

Open akjt opened 1 year ago

akjt commented 1 year ago

I see the example regarding centerpoint scn 3D backbone which includes fusion and QAT.

Does this library support the use of PTQ instead - and in case where can I look for an example?

superpigforever commented 1 year ago

https://github.com/NVIDIA-AI-IOT/Lidar_AI_Solution/blob/3a45a0fc1283f50772d5ea472f4421e9564a8cd4/CUDA-CenterPoint/qat/tools/centerpoint_qat_pipline.py#L216

Set ptq_mode in the args to True

akjt commented 1 year ago

But is that not only for training? I already have a trained model, and wanted to use the fusion+quantisation - so i am using this centerpoint_eval.py , however, the problem is that it wants to fuse before loading the weights, which won't work for PTQ scenario since you dont compress the weights. It works well when I just apply fusion since it would load the weights before the layer fusions.

hopef commented 1 year ago

PTQ is the first step of QAT. You can modify it yourself to achieve your purpose. We need to sort out the relationship between PTQ and layer-fuse and handle it properly, as done in the code.