mmasana
commented
1 year ago Hi!
Could you share the arguments that you used for your setting? That would make it easier to figure out the discrepancy.
Maybe you didn't run the experiment with the gridsearch parameters and directly used lambda=1 for all tasks?
Here is the output for one of the seeds:
TAg Acc
78.8% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Avg.: 78.8%
72.2% 41.9% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Avg.: 57.0%
57.5% 24.3% 70.6% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Avg.: 50.8%
55.8% 23.3% 70.5% 12.8% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Avg.: 40.6%
45.6% 12.6% 57.3% 5.7% 74.2% 0.0% 0.0% 0.0% 0.0% 0.0% Avg.: 39.1%
41.4% 10.9% 52.9% 2.6% 68.2% 44.1% 0.0% 0.0% 0.0% 0.0% Avg.: 36.7%
36.2% 8.8% 51.1% 1.2% 64.8% 38.0% 49.3% 0.0% 0.0% 0.0% Avg.: 35.6%
33.0% 7.7% 49.6% 1.1% 64.3% 32.4% 45.9% 36.1% 0.0% 0.0% Avg.: 33.8%
25.8% 4.7% 45.7% 0.9% 58.4% 26.6% 36.8% 26.0% 58.2% 0.0% Avg.: 31.5%
21.8% 5.2% 44.1% 0.8% 57.3% 25.5% 32.9% 23.7% 58.8% 43.0% Avg.: 31.3% and the relevant/simplified arguments used:
"T" : 2,
"approach" : "lwf",
"batch_size" : 128,
"datasets" : ["cifar100_icarl"],
"gridsearch_tasks" : 10,
"nepochs" : 200,
"network" : "resnet32",
"num_exemplars" : 0,
"num_exemplars_per_class" : 0,
"num_tasks" : 10,
"pretrained" : false,
"seed" : 0,
"warmup_nepochs" : 0,
"weight_decay" : 0.0
Hello, I noticed that the accuracy of the LwF method in CIFAR-100 (10/10) drops to 16% after 10 tasks when the No exemplars approach is used, which is different fromthe 30.2% accuracy reported in your paper. I waswondering if you could help me understand this discrepancy? Thank you.