search

wangzy22 / P2P

[NeurIPS 2022 Spotlight] P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting
https://p2p.ivg-research.xyz
MIT License
126 stars 9 forks source link

What's the meaning of `offset` in `ProjEnc`? #1

Closed Silverster98 closed 2 years ago

Silverster98 commented 2 years ago

I notice there is a variable offset defined in the ProjEnc class at Line33. And it is used for computing idx_xy_dense at Line65. BTW, what's the meaning of idx_xy_dense and f_dense? I didn't find related clarification in the original paper. Can you explain more?

https://github.com/wangzy22/P2P/blob/b26e9942a1bafd70e999815df5feb7d1a3210b4c/models/layers/encoder.py#L33

https://github.com/wangzy22/P2P/blob/b26e9942a1bafd70e999815df5feb7d1a3210b4c/models/layers/encoder.py#L65

https://github.com/wangzy22/P2P/blob/b26e9942a1bafd70e999815df5feb7d1a3210b4c/models/layers/encoder.py#L81

wangzy22 commented 2 years ago

Hi, thanks for your interest in our paper!

The offset variable is designed to project a point into 3x3 pixels that form a square centered at the original projection pixel. Given that the point cloud is sparse, we believe that this densify operation would produce smoother image. Therefore, idx_xy_dense is the densified index for projection and f_dense is just a duplication of the original feature to match size.