OpenRobotLab / PointLLM

[ECCV 2024] PointLLM: Empowering Large Language Models to Understand Point Clouds
https://runsenxu.com/projects/PointLLM
450 stars 22 forks source link

Training of Point-BERT #7

Closed AndreAmaduzzi closed 8 months ago

AndreAmaduzzi commented 8 months ago

Hello! I have a question regarding the training of Point-BERT. In your paper, you claim what follows:

As the original implementation of ULIP-2 only supports point clouds with spatial coordinates (xyz), we re-train Point-BERT with color information (xyzrgb), following the same procedure outlined in the ULIP-2 paper. For training Point-BERT, we employ ViT-L/14 from OpenCLIP [20] and use point clouds from the Cap3D [29] dataset...

I am wondering if you trained Point-BERT from scratch, by following ULIP-2 pipeline (aligning 3D features with 2D and text features) or finetuned it starting from a Point-BERT model trained on the reconstruction task, as the original Point-BERT paper suggests.

Thanks in advance, Andrea

RunsenXu commented 8 months ago

Train from scratch.