Implements a new training script for training a classification head. The new nn.module in model.py is separate from other components of M4T and takes speech encoder output and maps it into language probabilities with softmax. All layers of M4T are frozen to prevent training of those components. A new dataset.py script is implemented for downloading common voice from hugging face.
Most recent results from training:
Samples in train dataset:
- eng - 647 samples
- fra - 676 samples
- deu - 862 samples
- rus - 775 samples
- spa - 908 samples
- hin - 418 samples
- Total - 4286 samples
Samples in eval dataset:
- eng - 394 samples
- fra - 289 samples
- deu - 363 samples
- rus - 356 samples
- spa - 408 samples
- hin - 239 samples
- Total - 2049 samples
Implements a new training script for training a classification head. The new nn.module in model.py is separate from other components of M4T and takes speech encoder output and maps it into language probabilities with softmax. All layers of M4T are frozen to prevent training of those components. A new dataset.py script is implemented for downloading common voice from hugging face.
Most recent results from training:
Samples in train dataset:
Samples in eval dataset:
Parameters:
I am collaborating with @zrthxn on this project