felixbur / nkululeko

Machine learning speaker characteristics
MIT License
31 stars 5 forks source link

Make base model for finetuning as variable for INI file #123

Closed bagustris closed 4 months ago

bagustris commented 4 months ago

With this PR, related to #117, I modified model_tuned.py so the base model can be configured in the INI file. New value for INI file:

pretrained_model = microsoft/wavlm-base

Without specifying pretrained_model, the default facebook/wav2vec2-large-robust-ft-swbd-300h will be used. Example of results

# wav2vec2 large robust swbd (default)
DEBUG reporter: epoch: 10, UAR: .46, (+-.422/.496), ACC: .515
DEBUG reporter: labels: ['happiness', 'neutral', 'anger', 'sadness', 'fear', 'boredom', 'disgust']
DEBUG reporter: result per class (F1 score): [0.625, 0.508, 0.074, 0.0, 0.0, 0.568, 0.929]
WARNING experiment: Save experiment: Can't pickle the trained model so saving without it. (it should be stored anyway)
DEBUG experiment: Done, used 1633.975 seconds
# model_ckpt = facebook/hubert-large-ll60k
BUG reporter: epoch: 10, UAR: .633, (+-.589/.681), ACC: .662
DEBUG reporter: labels: ['happiness', 'neutral', 'anger', 'sadness', 'fear', 'boredom', 'disgust']
DEBUG reporter: result per class (F1 score): [0.743, 0.536, 0.8, 0.533, 0.054, 0.712, 0.897]
WARNING experiment: Save experiment: Can't pickle the trained model so saving without it. (it should be stored anyway)
DEBUG experiment: Done, used 1629.633 seconds
# model_ckpt = microsoft/wavlm-base
BUG reporter: epoch: 10, UAR: .802, (+-.753/.843), ACC: .826
DEBUG reporter: labels: ['happiness', 'neutral', 'anger', 'sadness', 'fear', 'boredom', 'disgust']
DEBUG reporter: result per class (F1 score): [0.824, 0.861, 0.92, 0.871, 0.316, 0.852, 0.982]
WARNING experiment: Save experiment: Can't pickle the trained model so saving without it. (it should be stored anyway)
DEBUG experiment: Done, used 758.234 seconds