Reproducibility issue... ResNet 32 vs 34.. Fraud?

[piltgha]

Thank you for your efforts, but I still doubt about the performance you reported since the Colab code uses ResNet-34 but your paper achieved the performance with ResNet32.

ResNet 34 has far different from ResNet 32 in the perspective of internal channel. ResNet 32 uses 3 stage with internal channel 16, 32, 64. and final fully connected layer uses only 64-dim feature.

ResNet 34 uses 512-dim feature with very large internal channels begins from 64.

Not just piling up 2 layers, they have such a big difference. But you only suggested ResNet34, which has huge parameters. But you reported you used ResNet 32 for CIFAR 100-LT to achieve the performance.

[CVHvn]

Maybe they increase the features of ResNet 32 from 16, 32, 64 to 64, 128, 256. This is still "ResNet 32" :( but have a same size with ResNet 34.

[piltgha]

Then did you used your resnet32 for other methods' evaluation also?

2023년 4월 25일 (화) 오전 1:23, CVHvn @.***>님이 작성:

Maybe they increase the feature of ResNet 32 from 16, 32, 64 to 64, 128,

256. This is still "ResNet 32" :( but have a same size with Resnet 34.

— Reply to this email directly, view it on GitHub https://github.com/ShadeAlsha/LTR-weight-balancing/issues/16#issuecomment-1520477943, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJULK3JYYG7FDVNGMBQAUB3XC2SHXANCNFSM6AAAAAAWSAN7LY . You are receiving this because you authored the thread.Message ID: @.***>

[CVHvn]

I don't know, I am not author. I ask author of other github about same issue (ResNet 32 vs ResNet 34): https://github.com/ynu-yangpeng/GLMC/issues/1

[piltgha]

Got it.

still it seems to be unfair.

2023년 4월 25일 (화) 오전 11:51, CVHvn @.***>님이 작성:

I don't know, I am not author. I ask author of other github about same issue (ResNet 32 vs ResNet 34): ynu-yangpeng/GLMC#1 https://github.com/ynu-yangpeng/GLMC/issues/1

— Reply to this email directly, view it on GitHub https://github.com/ShadeAlsha/LTR-weight-balancing/issues/16#issuecomment-1521087359, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJULK3PIDZAMVSYOGV4UPPLXC432HANCNFSM6AAAAAAWSAN7LY . You are receiving this because you authored the thread.Message ID: @.***>

[sehunfromdaegu]

Hmm, I cannot get similar scores in the paper, with the same hyperparameters in the notebook with resnet32.