search

HobbitLong / RepDistiller

[ICLR 2020] Contrastive Representation Distillation (CRD), and benchmark of recent knowledge distillation methods
BSD 2-Clause "Simplified" License
2.19k stars 400 forks source link

about using the resnet models for cifar10 #48

Open EmnaGuermazi97 opened 2 years ago

EmnaGuermazi97 commented 2 years ago

Hello Sir, I would like to start by saying how great this work is! And I would like to know if the resnet models can be used directly on cifar10 instead of cifar100 or if there is some modification I should make? I tried using KD (logits) to make resnet32 learn from resnet110 but unfortunately, the accuracy I get is pretty bad: Acc= 43.35 after 250 epochs, a=0.1, b= 0.9 with no modification in the resnet models nor in function adjust_learning_rate.

ShristiDasBiswas commented 7 months ago

Hi, were you able to get good accuracy after modifications?