Add a documentation page for data quality required for fine-tuning

Aml-Hassan-Abd-El-hamid commented 5 days ago

Self Checks

[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

I'm trying to fine-tune the model to be able to pronounce Egyptian dialect.

I currently have a number of long videos -between 6 to 8 hours- that contain Egyptian books and the corresponding audio for different people reading those books, I'm cutting those audios into segments on silence and matching the segments to the text from the books, but I'm lacking some information to do so, such as:

How long should the ideal audio/text segments be to get the best results?
Should I keep the audio stereo or should I turn it to the mono channel?
Should I resample those audios or keep their original frequency?
should I delete the audio segments with slight background music or should I keep them?
should I keep the punctuation in the text or should I delete them?
Is there any cleaning for the text or the audio that should be done before fine-tuning?

2. Additional context or comments

No response

3. Can you help us with this feature?

[X] I am interested in contributing to this feature.

PoTaTo-Mika commented 4 days ago

In the later version, we plan to remove the fine-tune part. Instead, we'll add a series of tools to enhance your reference audio's quality.

Aml-Hassan-Abd-El-hamid commented 4 days ago

But what if I need to add a new language or a dialect that the model usually doesn't handle? We need to fine-tune the model to accomplish such a task, right?

PoTaTo-Mika commented 4 days ago

But what if I need to add a new language or a dialect that the model usually doesn't handle? We need to fine-tune the model to accomplish such a task, right?

True, If you want to fine-tune for a new language (though the next version will support most of spoken languages in the world) ,you may need about 2K hours of low quality data, and about 100h (the more, the better) high quality data (44.1khz with high accuracy label). Hope that this can help you with running the project.

Aml-Hassan-Abd-El-hamid commented 4 days ago

Thanks a lot for your response, that's really helpful, I have one last question: does the data need to be cut to a certain length? I have multiple long audios -around 7 to 8 hours each- should I cut them down to shorter segments? and If I should do so, what is the recommended segment length? 15 minutes? 5 minutes? 30 seconds?

PoTaTo-Mika commented 4 days ago

Thanks a lot for your response, that's really helpful, I have one last question: does the data need to be cut to a certain length? I have multiple long audios -around 7 to 8 hours each- should I cut them down to shorter segments? and If I should do so, what is the recommended segment length? 15 minutes? 5 minutes? 30 seconds?

Yes, we recommend you to cut them into 30s / per segment.

Aml-Hassan-Abd-El-hamid commented 4 days ago

Thank you very much for your helpful and fast responses

fishaudio / fish-speech