About the instruct fine tuning

FunAudioLLM / CosyVoice

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

https://funaudiollm.github.io/

Apache License 2.0

4.66k stars 471 forks source link

About the instruct fine tuning #149

Open Ozawakun97 opened 1 month ago

Ozawakun97 commented 1 month ago

Hello dear Tongyi SpeechTeam I am interested with the controllable generation via instruction, and I want to fine-tune the model with my own version of data. Based on this, my question is, are there examples of the data for fine-tuning? Any requirements of the formatting? Besides, can the train.py script directly employed with fine-tuning? Are there any modifications to the script? Looking forward to your reply

traddo commented 1 month ago

据我实验，创建自有spk2info.pt，比微调效率高。

Ozawakun97 commented 1 month ago

据我实验，创建自有spk2info.pt，比微调效率高。

创建自有的spk2info.pt 然后直接inference吗

traddo commented 1 month ago

据我实验，创建自有spk2info.pt，比微调效率高。

创建自有的spk2info.pt 然后直接inference吗

对，也可以使用自有spk2info进行instruct_inference

ScottishFold007 commented 1 month ago

据我实验，创建自有spk2info.pt，比微调效率高。

会不稳定，韵律也经常出错，指望zero shot出能上线用的活、不经过一定数据的微调，无异于痴人说梦

traddo commented 1 month ago

据我实验，创建自有spk2info.pt，比微调效率高。

会不稳定，韵律也经常出错，指望zero shot出能上线用的活、不经过一定数据的微调，无异于痴人说梦

是，但这是比较简单的方法，修改一下文字中奇奇怪怪的标点符号，会好很多。

yangcunning1 commented 1 month ago

据我实验，创建自有spk2info.pt，比微调效率高。

怎么创建自有的spk2info.pt啊？