search

SHI-Labs / OneFormer

OneFormer: One Transformer to Rule Universal Image Segmentation, arxiv 2022 / CVPR 2023
https://praeclarumjj3.github.io/oneformer
MIT License
1.46k stars 129 forks source link

Metrics are zero during evaluation #39

Closed sahityabonumaddi closed 1 year ago

sahityabonumaddi commented 1 year ago
Thanks for your excellent work. I have used pretrained weights of swin backbone and evaluated the model of ADE20k and got he following results. 03/25 23:29:06 d2.evaluation.panoptic_evaluation]: Panoptic Evaluation Results: PQ SQ RQ #categories
All 0.000 0.000 0.000 150
Things 0.000 0.000 0.000 100
Stuff 0.000 0.000 0.000 50

[03/25 23:46:54 d2.evaluation.testing]: copypaste: Task: sem_seg [03/25 23:46:54 d2.evaluation.testing]: copypaste: mIoU,fwIoU,mACC,pACC [03/25 23:46:54 d2.evaluation.testing]: copypaste: 0.0029,0.0005,0.2547,0.0320 [03/25 23:46:54 d2.evaluation.testing]: copypaste: Task: panoptic_seg [03/25 23:46:54 d2.evaluation.testing]: copypaste: PQ,SQ,RQ,PQ_th,SQ_th,RQ_th,PQ_st,SQ_st,RQ_st [03/25 23:46:54 d2.evaluation.testing]: copypaste: 0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000,0.0000 [03/25 23:46:54 d2.evaluation.testing]: copypaste: Task: bbox [03/25 23:46:54 d2.evaluation.testing]: copypaste: AP,AP50,AP75,APs,APm,APl [03/25 23:46:54 d2.evaluation.testing]: copypaste: 0.0000,0.0000,0.0000,0.0000,0.0000,0.0000 [03/25 23:46:54 d2.evaluation.testing]: copypaste: Task: segm [03/25 23:46:54 d2.evaluation.testing]: copypaste: AP,AP50,AP75,APs,APm,APl [03/25 23:46:54 d2.evaluation.testing]: copypaste: 0.0000,0.0000,0.0000,0.0000,0.0000,0.0000

Anyhelp would be appreciated. Thanks in advance !

praeclarumjj3 commented 1 year ago

Hi @sahityabonumaddi, you evaluated OneFormer after only loading the pretrained (on classification) backbone weights. You need to load the checkpoint jointly trained for the segmentation task. The checkpoints are given here.