search

Alibaba-MIIL / ML_Decoder

Official PyTorch implementation of "ML-Decoder: Scalable and Versatile Classification Head" (2021)
MIT License
315 stars 52 forks source link

Doubt in the performance on ResNet backbone #29

Closed yzqinjacob closed 2 years ago

yzqinjacob commented 2 years ago

Hi! Thanks a lot for sharing this valuable work. But I have some questions for the performance of ML_decoder in ResNet backbone. It seems that you haven't released the RessNet version in your code but some relevant datas appear in your paper. I just exchange the TResNet into ResNet but the mAP score is only 81.1, so far from your released score 87.1. So I wanna ask you whether the experimental settings on ResNet and TResNet are different or I mistakenly combine ResNet with ML_decoder, it will be very nice of you to realese the function of ResNet version. Thanks again!

mrT23 commented 2 years ago

We have tested ResNet with ML-Decoder in the paper on single-label classification (figure 6), it works very well. results of ResNet on MS-COCO we have in ASL paper.

I am not sure what configuration you ran, or even which ResNet (18, 50, 101 ??). anyway - why the big score gap: 1) TResNet are dramatically better models than plain ResNet 2) We are you using good pretrain for TResNet (imageNet 21K or OpenImages), while you are probably using a bad old 1K pretrain.

hence, i wont release ResNet models. move to newer and better models, like TResNet.