search

sahagobinda / GPM

Official [ICLR] Code Repository for "Gradient Projection Memory for Continual Learning"
MIT License
86 stars 16 forks source link

About the code #2

Closed DLwbm123 closed 3 years ago

DLwbm123 commented 3 years ago

Hi! I have a question about the function get_representation_matrix. Why was the batch_list set to [2 * 12, 100, 100, 125, 125]? And can I set it to another batch_list? Thanks!

sahagobinda commented 3 years ago

For CIFAR experiments, the number of samples (n_s) used for representation matrix construction for SVD was chosen to be 125. It is a hyperparameter (please see Table 6 in the paper). In the earlier layers, to keep the representation matrix size reasonable we used a subset of these 125 samples as reflected in the batch_list. Depending on your problem and experimental setup you can try out different values in the batch_list.