search

JAYATEJAK / GVAlign

Robust Feature Learning and Global Variance-Driven Classifier Alignment for Long-Tail Class Incremental Learning
4 stars 0 forks source link

Reproducibility of the code #6

Open Srinivasa-Divakar-Bhat opened 3 months ago

Srinivasa-Divakar-Bhat commented 3 months ago

Thanks a lot for making the code public. I had a couple of questions regarding reproducing the results in your paper.

  1. When I try to reproduce the results for cifar 100 shuffled and ordered 5 task experiments, I am getting the following results (I am assuming the accuracy given in your paper is the TAg Acc). For the cifar100 shuffled experiments image For cifar100 ordered experiments image

The numbers given in the paper 47.13% and 42.80%. Can you please let me know why is this so? Also is there anyway I can reproduce the same accuracy numbers as in the paper.

  1. There seems to be some issue with the ImageNet training. I am getting this error when I try to run the experiment, can you help me resolve it. image

Thanks in advance.

JAYATEJAK commented 3 months ago

Hi @Srinivasa-Divakar-Bhat,

  1. It is average incremental accuracy; it is the standard metric to report for CIL tasks and is calculated as (47.3 + 41.7 + 43.9 + 40.7 + 40.4 + 39.8)/6. (Avg of all cumulative accuracies).
  2. Maybe there is a problem in splitting classes.