search

Julie-tang00 / Point-BERT

[CVPR 2022] Pre-Training 3D Point Cloud Transformers with Masked Point Modeling
MIT License
542 stars 65 forks source link

Confusion about the Point Tokenizer #28

Closed auniquesun closed 2 years ago

auniquesun commented 2 years ago

@yuxumin @lulutang0608 @raoyongming Thanks for sharing the paper and code.

One point I am confused with is the Point Tokenizer in the framework.

According to the paper and my understanding, the farthest point sampling (FPS) produces g centers, after that kNN is used such that there are g groups, then you adopt the mini-PointNet to extract featues of previous g groups and the output can be treated as an input sequence to standard Transformer.

Immediately after that, however, you give a small section on Point Tokenizer, it actually is a DGCNN. My question is what's the utility of Point Tokenizer and why need to tokenize embeddings since the input sequences have been created and the inner embeddings are inherently separated by FPS according to previous Point Embeddings.

Is Point Tokenizer necessary in the framework?

Julie-tang00 commented 2 years ago

Thanks for being interested in our work. As for Point Tokenizer, which is used to generate the self-supervised signals for pre-training Transformer. More specifically, the Transformer's input is the input sequences (as you know), and the Transformer's output is the self-generated point tokens.

获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: Jerry Sun @.> 发送时间: Sunday, April 3, 2022 5:21:07 PM 收件人: lulutang0608/Point-BERT @.> 抄送: Lulu Tang @.>; Mention @.> 主题: [lulutang0608/Point-BERT] Confusion about the Point Tokenizer (Issue #28)

@yuxuminhttps://github.com/yuxumin @lulutang0608https://github.com/lulutang0608 @raoyongminghttps://github.com/raoyongming Thanks for sharing the paper and code.

One point I am confused with is the Point Tokenizer in the framework.

According to the paper and my understanding, the farthest point sampling produces g centers, after that kNN is used such that there are g groups, then you adopt the mini-PointNet to extract featues of previous g groups and the output can be treated as an input sequence to standart Transformer.

Immediately after that, however, you give a small section on Point Tokenizer. My question is what's the utility of Point Tokenizer and why is it neccessary since the input sequences have been ready according to previous Point Embeddings.

― Reply to this email directly, view it on GitHubhttps://github.com/lulutang0608/Point-BERT/issues/28, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVEZPPUP7V7IBRVYKFKADI3VDFPIHANCNFSM5SM5MPNQ. You are receiving this because you were mentioned.Message ID: @.***>

yuxumin commented 2 years ago

Close it since no response. Feel free to re-open it if problems still exist