ContinualAI / avalanche

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.
http://avalanche.continualai.org
MIT License
1.79k stars 292 forks source link

Confusion Matrix - BWT, FWT metrics #1667

Open loukasilias opened 1 month ago

loukasilias commented 1 month ago

Hello,

How can I have access to the confusion matrix per experience? I have two experiences.

I have this integrated the code below, but I dont know how to access confusion matrix.

eval_plugin = EvaluationPlugin(
    accuracy_metrics(minibatch=True, epoch=True, experience=True, stream=True),
    loss_metrics(minibatch=True, epoch=True, experience=True, stream=True),
    timing_metrics(epoch=True),
    bwt_metrics(experience=True, stream=True),
    forward_transfer_metrics(experience=True, stream=True),
    forgetting_metrics(experience=True, stream=True),
    cpu_usage_metrics(experience=True),
    confusion_matrix_metrics(num_classes=len(benchmark.train_stream[0].classes_in_this_experience), save_image=True, stream=True),
    disk_usage_metrics(minibatch=True, epoch=True, experience=True, stream=True),
    loggers=[InteractiveLogger()],
    strict_checks=False
)

In the end, I see the following: ConfusionMatrix_Stream/eval_phase/test_stream = <avalanche.evaluation.metric_results.AlternativeValues object at 0x7effe45ac550>

Also, when I use as metrics Forward Knowledge Transfer and Backward Knowledge Transfer, I get this error: image

Thank you.