search

arthurdouillard / incremental_learning.pytorch

A collection of incremental learning paper implementations including PODNet (ECCV20) and Ghost (CVPR-W21).
MIT License
388 stars 60 forks source link

Trying to incorporate podnet in ucir code #37

Closed RamMohan112 closed 3 years ago

RamMohan112 commented 3 years ago

In the classifier.py file while using cosine classifier

The value of both self.scaling and self.gamma remains 1 throughout the run. Could you briefly explain the use of both.

If we normalise both the features and weights their dot product will lie in -1 to 1 range so we need to scale them, which variable in your code is exactly doing that.

my run command is this -

python3 -minclearn --options options/podnet/podnet_cnn_cifar100.yaml options/data/cifar100_3orders.yaml \ --initial-increment 50 --increment 1 --fixed-memory \ --device --label podnet_cnn_cifar100_50steps \ --data-path <PATH/TO/DATA>

arthurdouillard commented 3 years ago

Hello,

About self.scaling

It's my fault the code isn't clear on that. I actually don't use the learned scalar of the CosineClassifier but use an external one managed by the BasicNet https://github.com/arthurdouillard/incremental_learning.pytorch/blob/master/inclearn/lib/network/basenet.py#L37 which is then used explicitely here https://github.com/arthurdouillard/incremental_learning.pytorch/blob/master/inclearn/models/podnet.py#L303 .

About self.gamma

It's just an hyperparameter that acts as a temperature in the merging of the modes. But it's defaulted to 1.