Unable to reproduce the performance in paper

hustvl / Featurized-QueryRCNN

Featurized Query R-CNN

https://arxiv.org/abs/2206.06258

MIT License

46 stars 3 forks source link

Unable to reproduce the performance in paper #1

Open chenzyhust opened 2 years ago

chenzyhust commented 2 years ago

Conifg: configs/queryrcnn.res50.100pro.3x.yaml Results: [07/02 18:39:24 d2.evaluation.testing]: copypaste: Task: bbox [07/02 18:39:24 d2.evaluation.testing]: copypaste: AP,AP50,AP75,APs,APm,APl [07/02 18:39:24 d2.evaluation.testing]: copypaste: 36.7532,53.7602,39.9495,22.2459,39.1920,46.0393

I use the default config, but the results is not right

mulinmeng commented 2 years ago

Conifg: configs/queryrcnn.res50.100pro.3x.yaml Results: [07/02 18:39:24 d2.evaluation.testing]: copypaste: Task: bbox [07/02 18:39:24 d2.evaluation.testing]: copypaste: AP,AP50,AP75,APs,APm,APl [07/02 18:39:24 d2.evaluation.testing]: copypaste: 36.7532,53.7602,39.9495,22.2459,39.1920,46.0393

I use the default config, but the results is not right

It should be that the model parameters are not fully loaded, I will verify it as soon as possible and update the model link here in time.

chenzyhust commented 2 years ago

I did not use the model link you gave, but retrained it myself.So can you train it again to verify it yourself?

mulinmeng commented 2 years ago

I did not use the model link you gave, but retrained it myself.So can you train it again to verify it yourself?

I just verified the results of the checkpoint and found no problems. You can check if the pretrained model (ImageNet) is loaded, or the batch-size is set correctly. You can give us feedback if you have any questions or results.

chenzyhust commented 2 years ago

After confirmation, it was found that the loaded pre-training model was different.“//ImageNetPretrained/torchvision/R-50.pkl” and "//ImageNetPretrained/MSRA/R-50.pkl".Is there any difference here?