search

CanPeng123 / FSCIL_ALICE

28 stars 6 forks source link

Harmonic Mean computation in the paper #10

Open noorahmedds opened 1 year ago

noorahmedds commented 1 year ago

Section 4.4 of the paper indicates that harmonic mean is calculated between the base session and "the average accuracy for the following incremental session classes". This calculation is not provided in the code base so I was wondering whether harmonic mean is calculated between base and all novel classes or just the novel classes from the current session.

CanPeng123 commented 1 year ago

Hi, thanks for your interest in our work. We calculated the harmonic mean between base classes and all novel classes from the first incremental step.

noorahmedds commented 1 year ago

Thank you for getting back to me quickly. Let me make sure I understand clearly. In the Mini ImageNet example, we calculate the harmonic mean by considering the accuracy of the base classes (labeled 0-59) and the accuracy of the novel classes only from the first stage of incremental learning (labeled 60-64). We then recalculate this same accuracy at the end of each incremental step.

CanPeng123 commented 1 year ago

Hi,

We calculate the harmonic accuracy differently. As we do not update the feature extractor after base session training, to better evaluate the prediction bias, we always calculate the harmonic mean between the base and the following classes. For example, for the MiniImage first step incremental setting (60 + 5), we calculate the harmonic mean between the base classes (labeled 0 -59) and all the new classes (labeled 60 - 64). Then, for the MiniImage second step incremental setting (60 + 5 +5), we calculate the harmonic mean between the base classes (labeled 0 -59) and all the new classes (labeled 60 - 69).

Hope this could clarify your confusion.