MingruiLiu-ML-Lab / Bilevel-Coreset-Selection-via-Regularization

[NeurIPS 2023] Bilevel Coreset Selection in Continual Learning: A New Formulation and Algorithm
MIT License
6 stars 1 forks source link

Question about the evaluation metrics FGT #2

Closed bhsimon0810 closed 3 weeks ago

bhsimon0810 commented 1 month ago

Hi, I am running experiments on Split CIFAR100 using the default setting, and get the following results: "AVG ACC": 56.25, "Forgetting": 7.0. I have noticed that there are some hyperparameters I should adjust from another issue. However, for the FGT metric, the difference between my result (7.0) and the one mentioned in the paper (0.051) is quite large. Could you please confirm if this significant difference (7 vs 0.051) can be reduced by simply increasing the "--outer_iter" from 5 to 10?