Fix Evaluation Logging in MOABB

[Drew-Wagner]

Fixes #24

• The final evaluation on the validation set using the averaged model was being logged as test resulting in confusing logs.

  Before:

  speechbrain.utils.checkpoints - Saved an end-of-epoch checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-24+02
  speechbrain.utils.epoch_loop - Going into epoch 10
  speechbrain.nnet.schedulers - Changing lr from 8.3e-06 to 9.7e-06
  speechbrain.utils.train_logger - epoch: 10, lr: 6.95e-06 - train loss: 8.12e-01 - valid loss: 1.39, valid f1: 1.11e-01, valid acc: 2.14e-01, valid cm: [[11  3  0  0]
  [13  1  0  0]
  [13  1  0  0]
  [14  0  0  0]]
  speechbrain.utils.checkpoints - Saved an end-of-epoch checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-24+03
  speechbrain.utils.checkpoints - Loading a checkpoint from results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-24+03
  speechbrain.utils.train_logger - epoch loaded: 10 - test loss: 1.39, test f1: 9.89e-02, test acc: 2.47e-01, test cm: [[71  1  0  0]
  [72  0  0  0]
  [72  0  0  0]
  [72  0  0  0]]
  speechbrain.utils.checkpoints - Saved an end-of-epoch checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-25+00
  speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-23+00
  speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-22+00
  speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-24+02
  speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-45+00
  speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-23+03
  speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-24+01
  speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-24+03
  speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-22+01
  speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-24+00
  speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-23+02
  speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-23+01
  speechbrain.utils.checkpoints - Loading a checkpoint from results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-52-25+00
  speechbrain.utils.train_logger - epoch loaded: 10 - test loss: 2.52e-01, test f1: 1.00e-01, test acc: 2.50e-01, test cm: [[14  0  0  0]
  [14  0  0  0]
  [14  0  0  0]
  [14  0  0  0]]

After:

speechbrain.utils.checkpoints - Saved an end-of-epoch checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-44+02
speechbrain.utils.epoch_loop - Going into epoch 10
speechbrain.nnet.schedulers - Changing lr from 8.3e-06 to 9.7e-06
speechbrain.utils.train_logger - epoch: 10, lr: 6.95e-06 - train loss: 8.20e-01 - valid loss: 1.39, valid f1: 1.26e-01, valid acc: 2.50e-01, valid cm: [[13  1  0  0]
 [13  1  0  0]
 [13  1  0  0]
 [14  0  0  0]]
speechbrain.utils.checkpoints - Saved an end-of-epoch checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-44+03
speechbrain.utils.checkpoints - Loading a checkpoint from results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-44+03
speechbrain.utils.train_logger - epoch loaded: 10 - test loss: 1.39, test f1: 1.58e-01, test acc: 2.67e-01, test cm: [[65  7  0  0]
 [60 12  0  0]
 [67  5  0  0]
 [68  4  0  0]]
speechbrain.utils.checkpoints - Saved an end-of-epoch checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-45+00
speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-44+02
speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-43+00
speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-44+00
speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-43+03
speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-43+02
speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-42+01
speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-44+01
speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-43+01
speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-42+00
speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-44+03
speechbrain.utils.checkpoints - Deleted checkpoint in results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-45-41+03
speechbrain.utils.checkpoints - Loading a checkpoint from results/MotorImagery/BNCI2014001/EEGNet_train/leave-one-session-out/sub-001/1test/save/CKPT+2024-03-23+09-48-45+00
speechbrain.utils.train_logger - epoch loaded: 10 - valid loss: 2.52e-01, valid f1: 1.11e-01, valid acc: 2.14e-01, valid cm: [[11  3  0  0]
 [13  1  0  0]
 [13  1  0  0]
 [14  0  0  0]]

[mravanelli]

Thank you @Drew-Wagner for the fix. I tested it and everything works well.