search

mmasana / FACIL

Framework for Analysis of Class-Incremental Learning with 12 state-of-the-art methods and 3 baselines.
https://arxiv.org/pdf/2010.15277.pdf
MIT License
524 stars 99 forks source link

can not reproduce the similar result as paper #28

Closed wangsen1312 closed 1 year ago

wangsen1312 commented 1 year ago

When I want to reproduce the results like figure 8 in the paper,

I run the code below: python3 -u src/main_incremental.py --approach ewc --num-tasks 10 --exemplar-selection herding --num-exemplars 2000 The results is different from the EWC-e in the figure 8? Any tips?

mmasana commented 1 year ago

Which results are you getting?

We provide a mean over 10 runs with different seeds and standard deviation. Are the results within the std?

Which trade-off value are you using? For our runs, we use the Continual Hyperparameter Framework (it can be called with the --gridsearch argument) so we can find the most optimal trade-off value with the information at hand (no use of previous or future data).

wangsen1312 commented 1 year ago

@mmasana Thank you for your quick response. I have tried with the gridsearch. I can almost reproduce the similar results now! Great work!