search

Julie-tang00 / Point-BERT

[CVPR 2022] Pre-Training 3D Point Cloud Transformers with Masked Point Modeling
MIT License
541 stars 65 forks source link

Questions about Point Patch Mixing #6

Closed haotian-liu closed 2 years ago

haotian-liu commented 2 years ago

Dear authors, thank you for the great work! I have some questions about Point Patch Mixing (PPM) introduced in your paper.

(1) Is PPM implemented with _mixup_pc(self, neighborhood, center, dvae_label) in the code?

(2) In the specific implementation, is the "virtual sample" in the paper generated by flipping the batch, i.e., i-th sample will be mixed with n-i-th sample, where n is the batch size?

Thank you!

yuxumin commented 2 years ago

Thanks for your interest in our work.

(1) Yes. (2) Yes.

Best!

haotian-liu commented 2 years ago

Thank you for confirming. This leaves me one confusion regarding this in the paper "In our implementation, we generate the same number of virtual samples as the real ones to make the pre-training task more challenging".

Could you please elaborate a bit about how we understand the number of "virtual samples" and "real samples". To my understanding, the current implementation is that the patch tokens are randomly selected and mixed (thus the ratio is also random, instead of the same).

Thanks!

yuxumin commented 2 years ago

Hi, @haotian-liu. The "number" here means the number of samples, not the number of tokens for one certain sample. For example, if batch_size is set to 32. We will generate another 32 samples during the MPM task. Then we get 64 samples within one forward. That is what the number of "virtual samples" and "real samples" mean.

haotian-liu commented 2 years ago

Thank you so much for clarifying!