search

RAIVNLab / supsup

Code for "Supermasks in Superposition"
117 stars 19 forks source link

GG experiments #11

Closed e7mul closed 3 years ago

e7mul commented 3 years ago

Thanks for the great effort to provide code for your paper!

I run into issue trying to reproduce your results on GG experiments for SplitCIFAR-100.

After cloning the repo and runing following command:

python experiments/GG/splitcifar100/rn18-supsup.py --gpu-sets="0|1|2|3" --data=/path/to/dataset/parent --seeds 1

the _adaptresults.csv file looks like this: (the accuracy for adapt which in this config is 'gt' are completely random but for last task)

12-09-20_19:12:19, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_0~numtaskslearned=20~tasknumber=0, RandSplitCIFAR100_0, 20, 0.8480, 0.2000 12-09-20_19:12:19, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_1~numtaskslearned=20~tasknumber=1, RandSplitCIFAR100_1, 20, 0.8280, 0.2000 12-09-20_19:12:19, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_2~numtaskslearned=20~tasknumber=2, RandSplitCIFAR100_2, 20, 0.9180, 0.2180 12-09-20_19:12:19, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_3~numtaskslearned=20~tasknumber=3, RandSplitCIFAR100_3, 20, 0.9320, 0.2000 12-09-20_19:12:20, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_4~numtaskslearned=20~tasknumber=4, RandSplitCIFAR100_4, 20, 0.8760, 0.1420 12-09-20_19:12:20, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_5~numtaskslearned=20~tasknumber=5, RandSplitCIFAR100_5, 20, 0.9500, 0.2000 12-09-20_19:12:20, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_6~numtaskslearned=20~tasknumber=6, RandSplitCIFAR100_6, 20, 0.9180, 0.2000 12-09-20_19:12:21, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_7~numtaskslearned=20~tasknumber=7, RandSplitCIFAR100_7, 20, 0.8100, 0.2000 12-09-20_19:12:21, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_8~numtaskslearned=20~tasknumber=8, RandSplitCIFAR100_8, 20, 0.8920, 0.2000 12-09-20_19:12:21, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_9~numtaskslearned=20~tasknumber=9, RandSplitCIFAR100_9, 20, 0.8680, 0.2240 12-09-20_19:12:21, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_10~numtaskslearned=20~tasknumber=10, RandSplitCIFAR100_10, 20, 0.9500, 0.2140 12-09-20_19:12:22, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_11~numtaskslearned=20~tasknumber=11, RandSplitCIFAR100_11, 20, 0.8040, 0.2000 12-09-20_19:12:22, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_12~numtaskslearned=20~tasknumber=12, RandSplitCIFAR100_12, 20, 0.9000, 0.2040 12-09-20_19:12:22, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_13~numtaskslearned=20~tasknumber=13, RandSplitCIFAR100_13, 20, 0.8880, 0.2020 12-09-20_19:12:22, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_14~numtaskslearned=20~tasknumber=14, RandSplitCIFAR100_14, 20, 0.8820, 0.1940 12-09-20_19:12:23, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_15~numtaskslearned=20~tasknumber=15, RandSplitCIFAR100_15, 20, 0.9120, 0.2000 12-09-20_19:12:23, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_16~numtaskslearned=20~tasknumber=16, RandSplitCIFAR100_16, 20, 0.8940, 0.2020 12-09-20_19:12:23, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_17~numtaskslearned=20~tasknumber=17, RandSplitCIFAR100_17, 20, 0.9160, 0.2000 12-09-20_19:12:23, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_18~numtaskslearned=20~tasknumber=18, RandSplitCIFAR100_18, 20, 0.9280, 0.2140 12-09-20_19:12:24, id=supsup~seed=0~sparsity=16~try=1~task=RandSplitCIFAR100_19~numtaskslearned=20~tasknumber=19, RandSplitCIFAR100_19, 20, 0.8100, 0.8100

vkramanuj commented 3 years ago

Hey thanks for raising this issue. In experiments/GG/splitcifar100/configs/rn18-supsup.yaml, can you change the bn_type from NonAffineBN to MultitaskNonAffineBN, and then let us know if that fixes this issue for you with the same command?

e7mul commented 3 years ago

Yes, it solved the problem, thanks!