search

mlzxy / devit

CoRL 2024
https://mlzxy.github.io/devit
MIT License
330 stars 45 forks source link

Why are the shapes of these two files different? ycb_prototypes.pth VS OVD prototypes #30

Open Anymake opened 10 months ago

Anymake commented 10 months ago

Why are the shapes of these two files different? ycb_prototypes.pth ：torch.Size([34, 1024]) weights/initial/open-vocabulary/prototypes/lvis/lvis_v1_novel_train.vitl14.pkl ：torch.Size([337, 10, 1024]) weights/initial/few-shot/prototypes/fs_coco_trainval_novel_5shot.vitl14.pkl ：torch.Size([20, 10, 1024]) I know dim0 is class num, but why the prototype of one class in demo is 1x1024, but in the eval is 10 x 1024 ?

elE0710 commented 5 months ago

Hi there,

the OVD prototypes are not mean values, while the ycb_prototypes are already meaned. In the devit/detectron2/modeling/meta_arch/devit.py the ovd prototypes get meaned and become the same size as the ycb_prototypes. Hope this helps