search

ZrrSkywalker / Point-NN

[CVPR 2023] Parameter is Not All You Need: Starting from Non-Parametric Networks for 3D Point Cloud Analysis
MIT License
477 stars 50 forks source link

Thank you for your wonderful work, could you please share more detail manners of plugging the PointNN into 3DETR that improve the performance? I'm interested in this, thank you. #3

Open RongkunYang opened 1 year ago

ZrrSkywalker commented 1 year ago

Sorry for my late reply. As illustrated in Section 3.2 of the paper, we only plug the point-memory bank of Point-NN into 3DETR, without the non-parametric encoder, for implementation efficiency. There are two steps:

  1. Point-Memory Bank. Given a pre-trained 3DETR, we first use it to detect the entire training set, and store the features of object queries, which are responsible for detecting the ground-truth objects. These features are regarded as Feature Memory, and their corresponding one-hot labels are regarded as Label Memory.
  2. Plug-and-Play. During inference, for each object query, we interpolate its classification prediction from the pre-trained MLP head with our prediction from point-memory bank. In this way, the complementarity of learning-based knowledge and similarity-based matching can improve the performance of 3DETR.

Thanks!