search

IDEA-Research / DINO

[ICLR 2023] Official implementation of the paper "DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection"
Apache License 2.0
2.15k stars 232 forks source link

Question about the training log #128

Open bartbuaa opened 1 year ago

bartbuaa commented 1 year ago

Hi, it is very impressive for the work of DINO. I have tried some times to repeat the training on COCO dataset with DINO-RS50 However, the MAP score seems not following well with your logs. I have also noticed that other researchers seem to repeat your experiment very well in #104. So it is so appreciated if some details about settings of training can be offered.

Comparisons among training logs for the first 11 epochs are shown here. yours: 0.230,0.332,0.380,0.400,0.414,0.430,0.431,0.446,0.454,0.458,0.462,0.490

104: 0.241,0.333,0.368,0.397,0.413,0.427,0.433,0.441,0.449,0.456,0.456,0.466

mine1: 0.255,0.344,0.376,0.400,0.409,0.422,0.424,0.427,0.430,0.432,0.430,0.441 mine2: 0.261,0.345,0.381,0.402,0.413,0.422,0.421,0.426,0.432,0.430,0.428,0.440

Here is the comparison between the first 20 epochs logs. It seems that my MAP score falls down after 5th epoch comparing with yours. yours: 0.230,0.332,0.380,0.400,0.414,0.430,0.431,0.446,0.454,0.458,0.462,0.490,0.466,0.469,0.468,0.476,0.477,0.473,0.481,0.480 mine: 0.261,0.345,0.381,0.402,0.413,0.422,0.421,0.426,0.432,0.430,0.428,0.440,0.438,0.434,0.432,0.431,0.427,0.424,0.423,0.418

The seed is set to 42 as default.

Thanks.

FengLi-ust commented 1 year ago

Have you changed anything in our code? The log is very different from ours. You may clone a clean code and rerun it, you are expected to get 48.5-49.0 in 12epochs. You can also check whether your COCO dataset is correct.

bartbuaa commented 1 year ago

Have you changed anything in our code? The log is very different from ours. You may clone a clean code and rerun it, you are expected to get 48.5-49.0 in 12epochs. You can also check whether your COCO dataset is correct.

So grateful for your reply. I will clone a clean code and release the new log here. Please maintain the issue opened for a little while. I will try it as soon as possible. Thanks.

bartbuaa commented 1 year ago

@FengLi-ust May I ask if you have any plans to release the best pre-trained model with MAP of 63.3? It improves a lot compared with DINO SWIN-L of 58.0 MAP score. Or could you offer some information about how to acquire such high MAP score? Have you ever tried to use a larger training set like Object365? It is so appreciated. Thanks.

FengLi-ust commented 1 year ago

Please refer to our previous issues with similar questions.

bartbuaa commented 1 year ago

Yes, details about how to get 63.3 MAP score are discussed in #57. Thanks.

bartbuaa commented 1 year ago

Have you changed anything in our code? The log is very different from ours. You may clone a clean code and rerun it, you are expected to get 48.5-49.0 in 12epochs. You can also check whether your COCO dataset is correct.

So grateful for your reply. I will clone a clean code and release the new log here. Please maintain the issue opened for a little while. I will try it as soon as possible. Thanks.

@FengLi-ust Hi, I have download new copies of the dataset as well as the code. The new log of dino rs50 seems pretty well just like yours. Thanks a lot.

yours: 0.230, 0.332, 0.380, 0.400, 0.414, 0.430, 0.431, 0.446, 0.454, 0.458, 0.462, 0.490

104: 0.241, 0.333, 0.368, 0.397, 0.413, 0.427, 0.433, 0.441, 0.449, 0.456, 0.456, 0.466

mine: 0.235, 0.332, 0.366, 0.399, 0.410, 0.427, 0.433, 0.438, 0.447, 0.454, 0.458, 0.487