search

chufanchen / read-paper-and-code

0 stars 0 forks source link

ICCV 2023 | SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model #158

Open chufanchen opened 5 months ago

chufanchen commented 5 months ago

https://github.com/GengDavid/SLCA

https://arxiv.org/abs/2303.05118

chufanchen commented 5 months ago

continual learning on a pre-trained model(CLPM)

There are two major strategies to leverage the pre-trained knowledge

  1. fine-tune the representations
  2. keep the representations fixed while learning a few additional parameters
chufanchen commented 5 months ago

Slow learner

using a much smaller learning rate ($0.0001$) for $θ{rps}$ and a slightly larger learning rate ($0.01$) for $θ{cls}$ can greatly enhance the performance of traditional baseline

The proposed SL can almost address the problem of the representation layer, yet the classification layer remains sub-optimal.

Slow Learner with Classifier Alignment