search

facebookresearch / seamless_communication

Foundational Models for State-of-the-Art Speech and Text Translation
Other
10.94k stars 1.06k forks source link

NotImplementedError: T2U finetuning implemented only for UnitYT2UModel why??!! #426

Open developeranalyser opened 7 months ago

developeranalyser commented 7 months ago

why??!!

%cd /content/my_seamless_communication/src

!torchrun \ --rdzv-backend=c10d \ --rdzv-endpoint=localhost:0 \ --nnodes=1 \ --nproc-per-node=1 \ /content/my_seamless_communication/src/finetune.py \ --mode SPEECH_TO_SPEECH \ --train_dataset /content/my_m4t_dataset/train_manifest.json \ --eval_dataset /content/my_m4t_dataset/validation_manifest.json \ --learning_rate 1e-6 \ --warmup_steps 100 \ --max_epochs 5 \ --patience 5 \ --batch_size 3 \ --save_model_to /content/my_m4t_dataset/checkpoint1.pth \ --model_name seamlessM4T_v2_large

yiyibooks commented 5 months ago

Hi @developeranalyser, I met the same issue. Would you please shed lights on how to solve this issue to finetune SPEECH_TO_SPEECH? Thank you very very much in advance.

developeranalyser commented 4 months ago

سلام و درود بله این کار برای مدل v2 الان امکان پذیر نیست و باید از V1 large استفاده کنید از کد من استفاده کنید بسیاری از توسعه ها و بهبود های حافظه https://github.com/developeranalyser/my_seamless_communication

yiyibooks commented 4 months ago

سلام و درود بله این کار برای مدل v2 الان امکان پذیر نیست و باید از V1 large استفاده کنید از کد من استفاده کنید بسیاری از توسعه ها و بهبود های حافظه https://github.com/developeranalyser/my_seamless_communication

Thank you so much!

RRThivyan commented 1 month ago

Hi, is your finetune model working well? Could anyone help me with finetuning for TTS? I am new to this and the steps provided, causing my model to error as some weights are missing.