search

facebookresearch / seamless_communication

Foundational Models for State-of-the-Art Speech and Text Translation
Other
10.8k stars 1.05k forks source link

Fixed finetuning trainer and enable freezing layers #418

Closed zrthxn closed 4 months ago

zrthxn commented 5 months ago

This fixes some issues with M4T finetuning CLI and enables freezing parts of a model before training

Example:

m4t_finetune \
  --train_dataset gigaspeech/xs/xs_train_manifest.json \
  --eval_dataset gigaspeech/xs/xs_validation_manifest.json \
  --batch_size 10 \
  --eval_steps 1000 \
  --learning_rate 0.00008 \
  --patience 10 \
  --save_model_to checkpoints/expt5_m4tM.pt \
  --model_name seamlessM4T_medium \
  --freeze_layers \
      model.speech_encoder_frontend \
      model.speech_encoder \
      model.text_encoder_frontend \
      model.text_decoder.layers.0 \
      model.text_decoder.layers.1 \
      model.text_decoder.layers.2