search

facebookresearch / VLPart

[ICCV2023] VLPart: Going Denser with Open-Vocabulary Part Segmentation
MIT License
348 stars 16 forks source link

Higher Evaluation Result on PACO #10

Closed ProvenceStar closed 8 months ago

ProvenceStar commented 9 months ago

Hi, I appreciate your great work, however, when I run evaluation on PACO dataset follow your instructions, I found out that both r_50 and swinbase model reach a higher AP than your paper on open-vocabulary task, the details are as follows:

r_50: AP AP50 AP75 APs APm APl APr APc APf post-processed
18.477 34.633 17.761 16.062 29.749 32.301 8.750 15.564 19.478 {'obj-AP': 31.371, 'obj-AP50': 53.012, 'obj-part-AP-heirarchical': 14.587, 'obj-part-AP50-heirarchical': 29.521}
AP AP50 AP75 APs APm APl APr APc APf post-processed
13.323 25.231 12.589 10.641 22.601 26.665 7.500 12.214 13.716 {'obj-AP': 27.955, 'obj-AP50': 46.029, 'obj-part-AP-heirarchical': 10.762, 'obj-part-AP50-heirarchical': 21.421}
Swinbase: AP AP50 AP75 APs APm APl APr APc APf post-processed
27.089 44.215 27.583 23.176 43.016 50.203 7.500 22.097 28.820 {'obj-AP': 45.284, 'obj-AP50': 62.063, 'obj-part-AP-heirarchical': 21.636, 'obj-part-AP50-heirarchical': 38.498}
AP AP50 AP75 APs APm APl APr APc APf post-processed
19.103 34.669 18.116 15.740 30.578 37.538 2.500 16.957 19.892 {'obj-AP': 37.66, 'obj-AP50': 56.677, 'obj-part-AP-heirarchical': 15.208, 'obj-part-AP50-heirarchical': 29.44}

and my evaluation command is: python train_net.py --config-file configs/paco/r50_paco.yaml --eval-only --num-gpus 8 and: python train_net.py --config-file configs/paco/swinbase_cascade_paco.yaml --eval-only --num-gpus 8

Could you please check your result or inform me that how should I reproduce your result, thanks!