Bias correction layer is different from the original paper

[sobir-git]

In the original paper, bias correction layer consists of only two parameters "alpha" and "beta", which are only applied to the logits corresponding to the last trained group of classes. Here it seems like you have multiple such layers for each group of classes. https://github.com/sairin1202/BIC/blob/56a34c47f4cce673aa646ecdb91030289e19fc84/trainer.py#L191-L202

[bwolfson97]

I was wondering about this too. I was confused between 2 ways of doing the bias correction layer:

1. Like this implementation
   • A new bias layer is added each increment. After learning a bias correction layer, it becomes a part of the model for all future increments.
   • So if there's 5 increments, at the end of training you'd have 5 different bias correction layers (i.e. 5 alphas and 5 betas)
2. Only 1 bias correction layer is learned
   • There is only ever 1 alpha and 1 beta. Each increment alpha and beta are relearned based on the new classes added in this increment.

If anyone has any insight into which is the correct version that'd be helpful.

[EdenBelouadah]

@bwolfson97 , I believe that there should be as many alphas and betas as incremental states

[bwolfson97]

@EdenBelouadah Thanks!

[wuyuebupt]

@bwolfson97 For each increment, you need to learn a new set of bias paremeters for the new added classes for sure. For the old bias parameters, you can keep it (this repo) or discard it after distillation, depanding on how you do the distillation. I feel that the results from the two ways should be similar.

[bwolfson97]

@wuyuebupt Awesome, thanks!