search

zhengye1995 / BLC

Background Learnable Cascade for Zero-shot object detection
Apache License 2.0
15 stars 2 forks source link

Not able to reproduce the results #2

Open akshitac8 opened 3 years ago

akshitac8 commented 3 years ago

Hello @zhengye1995,

Thank you for your work. I wanted to ask, after training the cascade semantic rcnn with the new W's which epoch weights do you use for computing ZSD and GZSD numbers?

Regards, Akshita

zhengye1995 commented 3 years ago

I use the last epoch (12 epoch) for inference.

akshitac8 commented 3 years ago

You mean, the latest.pth saved epoch weights?

zhengye1995 commented 3 years ago

Yes.

akshitac8 commented 3 years ago

I used the same latest epoch but still not able to reproduce the results.

zhengye1995 commented 3 years ago

Have you tried to use the weight I shared in readme for evaluation?

akshitac8 commented 3 years ago

I need to make changes to the architecture. So, I need to first reproduce the results by training from scratch. I ran this command

python tools/test.py configs/BLC/inference/gzsd/gzsd_cascade_semantic_rcnn_information_flow_learnable_bg_65_15_1x.py work_dirs/cascade_semantic_rcnn_r50_fpn_65_15_semantic_flow_learnable_bg_1x/epoch_12.pth --out results/gzsd_65_15.pkl

and these are the results I got,

ZSD - mAP | SEEN - mAP | UNSEEN - mAP | HM - mAP |   | ZSD - RE | SEEN - RE | UNSEEN - RE | HM - RE -- | -- | -- | -- | -- | -- | -- | -- | -- 13.4 | 36 | 13.3 | 19.6 |   | 50.24 | 58.56 | 49.06 | 53.39
sahilkhose commented 3 years ago

@akshitac8 were you able to reproduce the results by training from scratch?