search

yulu0724 / SDC-IL

Semantic Drift Compensation for Class-Incremental Learning (CVPR2020)
115 stars 34 forks source link

Why does the accuracy become lower after applying SDC? #4

Closed YandanYang closed 4 years ago

YandanYang commented 4 years ago

Hi yulu,

I have run your demo code, with the setting according to your paper:

DATASET=cifar100  
GPU=2
NET=resnet18 
LR=1e-6
EPOCH=51
SAVE=50
LOSS=triplet_no_hard_mining 
TASK=11 
BASE=50 
SEED=1993
for Method in LwF 
do
for Tradeoff in 1 
do

But the results after applying SDC is lower than the baseline, such as:

old mean: Weighted Accuracy of Model 10 is 0.146
SDC: Weighted Accuracy of Model 10 is 0.068

The degradation is the same with EWC, MAS, and finetune. I do not change the other part of your code. Do I miss some point in the code?

yulu0724 commented 4 years ago

I think there must be something wrong. Because after training 10 tasks, even using the old mean, the accuracy should be around 40%-45% with LwF. Make sure you use the right version of pytorch, use the warm-up model. Let me know if you still couldn't get the correct results.

YandanYang commented 4 years ago

Thanks. I use a different version of pytorch. I will check this again.