search

yuhangzang / OV-DETR

[Under preparation] Code repo for "Open-Vocabulary DETR with Conditional Matching" (ECCV 2022)
202 stars 20 forks source link

Performance not as expected #23

Open guangpintao opened 10 months ago

guangpintao commented 10 months ago

Using the code in repo and using the provided pre-trained model to test consistently with performance in paper, but when retraining, the metrics are only APall=51.3 APsee=59.7 APunseen=27.6, and the setting of parameter R is also different from that in the paper (3 in the paper and 12 in the code). What should be noted when reproducing the performance of the paper?

guangpintao commented 10 months ago

{"@timestamp":"2023-10-20 19:09:39.697","@message":" Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.329"} {"@timestamp":"2023-10-20 19:09:39.697","@message":" Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.513"} {"@timestamp":"2023-10-20 19:09:39.698","@message":" Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.346"} {"@timestamp":"2023-10-20 19:09:39.698","@message":" Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.196"} {"@timestamp":"2023-10-20 19:09:39.699","@message":" Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.364"} {"@timestamp":"2023-10-20 19:09:39.700","@message":" Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.449"} {"@timestamp":"2023-10-20 19:09:39.700","@message":" Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.275"} {"@timestamp":"2023-10-20 19:09:39.700","@message":" Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.494"} {"@timestamp":"2023-10-20 19:09:39.700","@message":" Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.557"} {"@timestamp":"2023-10-20 19:09:39.700","@message":" Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.364"} {"@timestamp":"2023-10-20 19:09:39.700","@message":" Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.593"} {"@timestamp":"2023-10-20 19:09:39.700","@message":" Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.735"} {"@timestamp":"2023-10-20 19:09:39.701","@message":"bbox AP seen: 59.74352140181269"} {"@timestamp":"2023-10-20 19:09:39.701","@message":"bbox AP unseen: 27.574026608034263"} {"@timestamp":"2023-10-20 19:09:42.705","@message":"Training time 3 days, 7:24:13"}