How to reproduce results on YOLOR-S4-DWT

[thanhnt-2658]

I'm trying to reproduce results on YOLOR-S4-DWT. It's reported 37% AP on paper branch. However, after several training times, the results around 35.1 and 35.2 AP. I use below command, is there something I need to change?:

python train.py --batch-size 32 --img 640 640 --data data/coco.yaml --cfg models/yolor-ssss-dwt.yaml --weights '' --device 0 --name yolor-ssss-dwt-baseline --hyp hyp.scratch.s.yaml --epochs 300

[WongKinYiu]

do you test with pycocotools?

[thanhnt-2658]

Yes, I do. I use the default evaluation method on this repo. The best AP reported on "results.txt" is ~35.1 AP.

[WongKinYiu]

AP reported on results.txt is not from pycocotools. You should make sure you put coco annotation json file at correct path and see the AP calculated by pycocotools. https://github.com/WongKinYiu/yolor/blob/paper/test.py#L245 Due to coco only select top 100 prediction for each category for evaluating mAP, the AP calculated by pycocotools usually 1~2% higher than the value reported on resluts.txt

[thanhnt-2658]

I did re-test and got 36.5 AP. Do you have more ideas to close the gap further?

[WongKinYiu]

you could check if your model goes into except block. https://github.com/WongKinYiu/yolor/blob/paper/models/common.py#L14 if you have not installed dwt, it will use reorg instead, which will gets a little bit lower AP.

[thanhnt-2658]

I accidentally trained two versions of YOLOR-S4-DWT on two machines, one used dwt and one used reorg. But the one with reorg achieves a little bit higher AP (0.1 AP).

[WongKinYiu]

in my previous experiments, dwt usually perform better than reorg.