I am not able to train a classifier with GenarativeReplay

[3koozy]

Hello guys , I hope everyone is doing alright :) I tried training an MLP classifier of SplitMNIST with GenerativeReplay from what I understood from the docs as follows:

---

# MODEL CREATION
model = SimpleMLP(num_classes=scenario.n_classes)
VAE_model = MlpVAE((1, 28, 28), nhid=2)

interactive_logger = InteractiveLogger()
logging = EvaluationPlugin(loss_metrics(minibatch=False, epoch=True, experience=True, stream=True),
                           accuracy_metrics(minibatch=False, epoch=True, experience=True, stream=True),
                           loggers=[interactive_logger])

# CREATE THE STRATEGY INSTANCE (GenerativeReplay)
VAE_strategy = VAETraining(
    VAE_model,
    torch.optim.Adam(VAE_model.parameters(), lr=0.001),
    train_mb_size=100,
    train_epochs=10
)

cl_strategy = SupervisedTemplate(
        model,
        torch.optim.Adam(model.parameters(), lr=0.001),
        CrossEntropyLoss(),
        train_mb_size=100,
        train_epochs=4,
        eval_mb_size=100,
        plugins=[GenerativeReplayPlugin(generator_strategy = VAE_strategy)],
        evaluator=logging,
    )

---

unfortunately, my classifier had 100% forgetting and it seems the generative Replay didn't work.

-- >> End of eval phase << --
    Loss_Stream/eval_phase/test_stream/Task000 = 16.9208
    Top1_Acc_Stream/eval_phase/test_stream/Task000 = 0.1090

---

can you help me make this example work, as the example provided in the docs, didn't work with a classifier?

[AntonioCarta]

We have an open PR from travela in the stable-continual-learning repository that should reproduce the results on SplitMNIST. You can check his parameters.

[3koozy]

thx this helps a lot. I managed to get to work on mnist but no luck so far on splitCIFAR10. perhaps a VAE model with some CNN layers can help ?

[AntonioCarta]

Unfortunately generative replay is hard to work with and there is not much we can do in Avalanche to improve this. Improving the VAE will surely improve the performance.