stakira / OpenUtau

Open singing synthesis platform / Open source UTAU successor
http://www.openutau.com
MIT License
2.06k stars 285 forks source link

DiffSinger Multi-Dictionary Support not working, or missing necessary information. #1272

Closed spicytigermeat closed 1 month ago

spicytigermeat commented 1 month ago

Acknowledgement

🐛 Describe the bug

I have trained a DiffSinger DB with the new multi-dictionary support, but I cannot for the life of me reproduce the results in PR#1248. The only way to get it to work is to create a replacement in the dsdict for every supported phoneme using the language prefix, but that prevents the phoneme hints from working.

See the attached photo showcasing the only way to get the phonemizer to not throw errors on every phoneme:

image

Again, doing the above causes phoneme hints to not be accepted without manually typing each language prefix.

See attached photo of my dsconfig for the duration model:

image

Do I have to declare the phonemes in the dsdict with or without the prefix?

Explains how to reproduce the bug

Use any DiffSinger VB trained with multi-dictionary support, and use them on any supported diffsinger phonemizer.

OS & Version

Windows 11

Logs

System.Exception: Phoneme "a" isn't supported by timing model. Please check D:\DIFFSINGER\test opu\OpenUtau-master\OpenUtau\bin\Debug\net6.0-windows\Singers\TIGER_DS_v107\dsdur\files/phonemes.json at OpenUtau.Core.DiffSinger.DiffSingerBasePhonemizer.PhonemeTokenize(String phoneme) in D:\DIFFSINGER\test opu\OpenUtau-master\OpenUtau.Core\DiffSinger\DiffSingerBasePhonemizer.cs:line 289 at OpenUtau.Core.DiffSinger.DiffSingerBasePhonemizer.b__31_1(dsPhoneme p) in D:\DIFFSINGER\test opu\OpenUtau-master\OpenUtau.Core\DiffSinger\DiffSingerBasePhonemizer.cs:line 329 at System.Linq.Enumerable.SelectEnumerableIterator`2.ToArray() at OpenUtau.Core.DiffSinger.DiffSingerBasePhonemizer.ProcessPart(Note[][] phrase) in D:\DIFFSINGER\test opu\OpenUtau-master\OpenUtau.Core\DiffSinger\DiffSingerBasePhonemizer.cs:line 327 at OpenUtau.Core.MachineLearningPhonemizer.SetUp(Note[][] groups, UProject project, UTrack track) in D:\DIFFSINGER\test opu\OpenUtau-master\OpenUtau.Core\MachineLearningPhonemizer.cs:line 39 at OpenUtau.Api.PhonemizerRunner.Phonemize(PhonemizerRequest request) in D:\DIFFSINGER\test opu\OpenUtau-master\OpenUtau.Core\Api\PhonemizerRunner.cs:line 102 Exception thrown: 'System.Exception' in OpenUtau.Core.dll

Elpatata145 commented 1 month ago

Im having the same issue

oxygen-dioxide commented 1 month ago

https://github.com/stakira/OpenUtau/pull/1248#issuecomment-2335117634 Here are instructions for packaging multi-dict diffsinger voicebank for openutau usage.

spicytigermeat commented 1 month ago

#1248 (comment) Here are instructions for packaging multi-dict diffsinger voicebank for openutau usage.

Thank you! This worked for me :)