search

salesforce / ULIP

BSD 3-Clause "New" or "Revised" License
436 stars 44 forks source link

About the pretained models of Pointbert #47

Open lulongfei-luffy opened 1 year ago

lulongfei-luffy commented 1 year ago

Thanks for sharing the code and pretrained models. I noticed that there are two checkpoint about pointbert, I wonder which one was trained on objaverse. does this pointbert_ULIP-2.pt) model trained on objaverse?

Tycho-Xue commented 9 months ago

yes, it's trained on Objaverse+shapenet

auniquesun commented 5 months ago

yes, it's trained on Objaverse+shapenet

@Tycho-Xue Thanks for sharing the pre-trained weights of ULIP-2. But I notice that there is huge gap between the previous version of ULIP-2 and the CVPR 2024 version of ULIP-2.

Specifically, in previous version of ULIP-2, the top-1 zero-shot acc. on ModelNet40 is 74.0%, but in the CVPR 2024 version, the acc. increases to 84.7%. I wonder what happened during this time?

I notice both the two versions use Objaverse as the pre-training set. The CVPR 2024 version seems having additional small-size ShapeNet as the pre-training dataset. Can it make such a big difference in zero-shot performance?

auniquesun commented 5 months ago

Thanks for sharing the code and pretrained models. I noticed that there are two checkpoint about pointbert, I wonder which one was trained on objaverse. does this pointbert_ULIP-2.pt) model trained on objaverse?

@lulongfei-luffy I think the released pointbert_ULIP-2.pt) is not the newest version since according to the CVPR 2024 ULIP-2, the top-1 zero-shot acc. of ULIP-2 on ModelNet40 is 84.7%, but in previous arXiv version of ULIP-2, the top-1 zero-shot acc. on the same dataset is 74.0%, which is 10.7% absolute decreasing.

Also, I wonder what happened during this time. What an amazing increase! From 74.0% to 84.7%.

Tycho-Xue commented 5 months ago

Hi @auniquesun , yeah, in the latest ULIP which is accepted by CVPR2024, I also scaled up the 3d backbone, CLIP backbones, added color, i've had this checkpoint for quite some time, just need some code changes to make it compatible with the current codebase. I'll find sometime to make the changes since it's approaching CVPR2024, I can just upload it first in case you need it urgently, it should be very straightforward to make the inference work.