search

Megvii-BaseDetection / GFSD

This project provides an implementation for "Generalized Few-Shot Object Detection without Forgetting" (CVPR2021) on PyTorch.
Apache License 2.0
46 stars 4 forks source link

Questions about results of Meta R-CNN #1

Closed woxue closed 3 years ago

woxue commented 3 years ago

Thanks for your interesting work! I have some uncertainty about the results of Meta R-CNN in your paper. They seems different from the original paper of Meta R-CNN. Is there any difference in the experimental setup? I notice that you say "Note that the results are not directly comparable because samples used for finetuning are different, which can make a significant impact on the final metrics." in your paper. So what makes the difference appear? (I thought both result follows the split created by Feature Reweighting ICCV19) Does star(*) after Meta R-CNN and FsDetView means that you reproduce their methods under your framework?

Results in your paper: image

Results in Meta R-CNN: image image

zb1439 commented 3 years ago

* indicates that these methods are reproduced under exactly the same samples to ensure comparability. Note that although we use the same class split, the samples used for fine-tuning are different (e.g. Meta R-CNN fine-tunes on randomly selected samples without fixing the random seed, and uses more than K-shot samples on base classes, which is different from our experimental setting), which makes a difference.

woxue commented 3 years ago

Perfectly clear! Thank you!

Wei-i commented 3 years ago

@zb1439 Hi, thanks for your great work, I also want to know why the mAP of FsDetView * drops a lot in your paper, while in FsDetView's paper, on COCO 10-shot, it can achieve 12.5. Is it the same as the difference of Meta R-CNN configuration? I would be grateful if you could apply for me.