search

uezo / ChatdollKit

ChatdollKit enables you to make your 3D model into a chatbot
Apache License 2.0
754 stars 76 forks source link

Need to switch from standard voice to Neural TTS voice in Azure TTS Loader #200

Open nakazax opened 1 year ago

nakazax commented 1 year ago

Currently, Azure TTS Loader is using the ja-JP-HarukaRUS voice, which is one of the pre-built standard voices. However, this voice has been retired, resulting in a 403 error when attempting to use it. To resolve this issue, we need to switch to a Neural TTS voice, such as ja-JP-NanamiNeural. You can find more information about Neural TTS voices at the following links:

Error message

Error occured in processing chat: HTTP/1.1 403 Forbidden
  at Cysharp.Threading.Tasks.UnityAsyncExtensions+UnityWebRequestAsyncOperationAwaiter.GetResult () [0x00066] in C:\0\unity\ChatdollKitDev\Assets\Plugins\UniTask\Runtime\UnityAsyncExtensions.cs:792 
  at ChatdollKit.Extension.Azure.AzureTTSLoader.DownloadAudioClipNativeAsync (System.String url, System.Byte[] data, System.Collections.Generic.Dictionary`2[TKey,TValue] headers, System.Threading.CancellationToken token) [0x000d7] in C:\0\unity\ChatdollKitDev\Assets\ChatdollKit\Extension\Azure\AzureTTSLoader.cs:98 
  at Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`2[TStateMachine,T].GetResult (System.Int16 token) [0x00000] in C:\0\unity\ChatdollKitDev\Assets\Plugins\UniTask\Runtime\CompilerServices\StateMachineRunner.cs:342 
  at ChatdollKit.Extension.Azure.AzureTTSLoader.DownloadAudioClipAsync (ChatdollKit.Model.Voice voice, System.Threading.CancellationToken token) [0x001a7] in C:\0\unity\ChatdollKitDev\Assets\ChatdollKit\Extension\Azure\AzureTTSLoader.cs:78 
  at Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`2[TStateMachine,T].GetResult (System.Int16 token) [0x00000] in C:\0\unity\ChatdollKitDev\Assets\Plugins\UniTask\Runtime\CompilerServices\StateMachineRunner.cs:342 
  at Cysharp.Threading.Tasks.UniTask`1+MemoizeSource[T].GetResult (System.Int16 token) [0x0006a] in C:\0\unity\ChatdollKitDev\Assets\Plugins\UniTask\Runtime\UniTask.cs:574 
  at Cysharp.Threading.Tasks.UniTask+WhenAnyLRPromise`1[T].TryLeftInvokeContinuation (Cysharp.Threading.Tasks.UniTask+WhenAnyLRPromise`1[T] self, Cysharp.Threading.Tasks.UniTask`1+Awaiter[T]& awaiter) [0x00000] in C:\0\unity\ChatdollKitDev\Assets\Plugins\UniTask\Runtime\UniTask.WhenAny.cs:116 
--- End of stack trace from previous location where exception was thrown ---

  at Cysharp.Threading.Tasks.UniTask+WhenAnyLRPromise`1[T].GetResult (System.Int16 token) [0x0000c] in C:\0\unity\ChatdollKitDev\Assets\Plugins\UniTask\Runtime\UniTask.WhenAny.cs:152 
  at ChatdollKit.Model.WebVoiceLoaderBase.WaitDownloadCancellable (Cysharp.Threading.Tasks.UniTask`1[T] downloadTask, System.Threading.CancellationToken cancellationToken) [0x0006c] in C:\0\unity\ChatdollKitDev\Assets\ChatdollKit\Scripts\Model\WebVoiceLoaderBase.cs:86 
  at Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1[TStateMachine].GetResult (System.Int16 token) [0x00000] in C:\0\unity\ChatdollKitDev\Assets\Plugins\UniTask\Runtime\CompilerServices\StateMachineRunner.cs:218 
  at ChatdollKit.Model.WebVoiceLoaderBase.GetAudioClipAsync (ChatdollKit.Model.Voice voice, System.Threading.CancellationToken cancellationToken) [0x0014c] in C:\0\unity\ChatdollKitDev\Assets\ChatdollKit\Scripts\Model\WebVoiceLoaderBase.cs:49 
  at Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`2[TStateMachine,T].GetResult (System.Int16 token) [0x00000] in C:\0\unity\ChatdollKitDev\Assets\Plugins\UniTask\Runtime\CompilerServices\StateMachineRunner.cs:342 
  at ChatdollKit.Model.ModelController.Say (ChatdollKit.Model.VoiceRequest request, System.Threading.CancellationToken token) [0x002b8] in C:\0\unity\ChatdollKitDev\Assets\ChatdollKit\Scripts\Model\ModelController.cs:364 
  at Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1[TStateMachine].GetResult (System.Int16 token) [0x00000] in C:\0\unity\ChatdollKitDev\Assets\Plugins\UniTask\Runtime\CompilerServices\StateMachineRunner.cs:218 
  at ChatdollKit.Model.ModelController.AnimatedSay (ChatdollKit.Model.AnimatedVoiceRequest request, System.Threading.CancellationToken token) [0x00143] in C:\0\unity\ChatdollKitDev\Assets\ChatdollKit\Scripts\Model\ModelController.cs:265 
  at Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1[TStateMachine].GetResult (System.Int16 token) [0x00000] in C:\0\unity\ChatdollKitDev\Assets\Plugins\UniTask\Runtime\CompilerServices\StateMachineRunner.cs:218 
  at ChatdollKit.Dialog.DialogController.OnPromptAsyncDefault (ChatdollKit.Dialog.DialogRequest dialogRequest, System.Threading.CancellationToken token) [0x00109] in C:\0\unity\ChatdollKitDev\Assets\ChatdollKit\Scripts\Dialog\DialogController.cs:252 
  at Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1[TStateMachine].GetResult (System.Int16 token) [0x00000] in C:\0\unity\ChatdollKitDev\Assets\Plugins\UniTask\Runtime\CompilerServices\StateMachineRunner.cs:218 
  at ChatdollKit.Dialog.DialogController.StartDialogAsync (ChatdollKit.Dialog.DialogRequest dialogRequest) [0x000d8] in C:\0\unity\ChatdollKitDev\Assets\ChatdollKit\Scripts\Dialog\DialogController.cs:309 
UnityEngine.Debug:LogError (object)
ChatdollKit.Dialog.DialogController/<StartDialogAsync>d__72:MoveNext () (at Assets/ChatdollKit/Scripts/Dialog/DialogController.cs:370)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1<ChatdollKit.Dialog.DialogController/<StartDialogAsync>d__72>:Run () (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:189)
Cysharp.Threading.Tasks.AwaiterActions:Continuation (object) (at Assets/Plugins/UniTask/Runtime/UniTask.cs:21)
Cysharp.Threading.Tasks.UniTaskCompletionSourceCore`1<Cysharp.Threading.Tasks.AsyncUnit>:TrySetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/UniTaskCompletionSource.cs:167)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1<ChatdollKit.Dialog.DialogController/<OnPromptAsyncDefault>d__70>:SetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:210)
ChatdollKit.Dialog.DialogController/<OnPromptAsyncDefault>d__70:MoveNext () (at Assets/ChatdollKit/Scripts/Dialog/DialogController.cs:252)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1<ChatdollKit.Dialog.DialogController/<OnPromptAsyncDefault>d__70>:Run () (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:189)
Cysharp.Threading.Tasks.AwaiterActions:Continuation (object) (at Assets/Plugins/UniTask/Runtime/UniTask.cs:21)
Cysharp.Threading.Tasks.UniTaskCompletionSourceCore`1<Cysharp.Threading.Tasks.AsyncUnit>:TrySetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/UniTaskCompletionSource.cs:167)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1<ChatdollKit.Model.ModelController/<AnimatedSay>d__50>:SetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:210)
ChatdollKit.Model.ModelController/<AnimatedSay>d__50:MoveNext () (at Assets/ChatdollKit/Scripts/Model/ModelController.cs:289)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1<ChatdollKit.Model.ModelController/<AnimatedSay>d__50>:Run () (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:189)
Cysharp.Threading.Tasks.AwaiterActions:Continuation (object) (at Assets/Plugins/UniTask/Runtime/UniTask.cs:21)
Cysharp.Threading.Tasks.UniTaskCompletionSourceCore`1<Cysharp.Threading.Tasks.AsyncUnit>:TrySetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/UniTaskCompletionSource.cs:167)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1<ChatdollKit.Model.ModelController/<Say>d__52>:SetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:210)
ChatdollKit.Model.ModelController/<Say>d__52:MoveNext () (at Assets/ChatdollKit/Scripts/Model/ModelController.cs:423)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1<ChatdollKit.Model.ModelController/<Say>d__52>:Run () (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:189)
Cysharp.Threading.Tasks.AwaiterActions:Continuation (object) (at Assets/Plugins/UniTask/Runtime/UniTask.cs:21)
Cysharp.Threading.Tasks.UniTaskCompletionSourceCore`1<UnityEngine.AudioClip>:TrySetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/UniTaskCompletionSource.cs:167)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`2<ChatdollKit.Model.WebVoiceLoaderBase/<GetAudioClipAsync>d__21, UnityEngine.AudioClip>:SetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:334)
ChatdollKit.Model.WebVoiceLoaderBase/<GetAudioClipAsync>d__21:MoveNext () (at Assets/ChatdollKit/Scripts/Model/WebVoiceLoaderBase.cs:67)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`2<ChatdollKit.Model.WebVoiceLoaderBase/<GetAudioClipAsync>d__21, UnityEngine.AudioClip>:Run () (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:313)
Cysharp.Threading.Tasks.AwaiterActions:Continuation (object) (at Assets/Plugins/UniTask/Runtime/UniTask.cs:21)
Cysharp.Threading.Tasks.UniTaskCompletionSourceCore`1<Cysharp.Threading.Tasks.AsyncUnit>:TrySetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/UniTaskCompletionSource.cs:167)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1<ChatdollKit.Model.WebVoiceLoaderBase/<WaitDownloadCancellable>d__23>:SetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:210)
ChatdollKit.Model.WebVoiceLoaderBase/<WaitDownloadCancellable>d__23:MoveNext () (at Assets/ChatdollKit/Scripts/Model/WebVoiceLoaderBase.cs:86)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1<ChatdollKit.Model.WebVoiceLoaderBase/<WaitDownloadCancellable>d__23>:Run () (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:189)
Cysharp.Threading.Tasks.AwaiterActions:Continuation (object) (at Assets/Plugins/UniTask/Runtime/UniTask.cs:21)
Cysharp.Threading.Tasks.UniTaskCompletionSourceCore`1<System.ValueTuple`2<bool, UnityEngine.AudioClip>>:TrySetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/UniTaskCompletionSource.cs:167)
Cysharp.Threading.Tasks.UniTask/WhenAnyLRPromise`1<UnityEngine.AudioClip>:TryLeftInvokeContinuation (Cysharp.Threading.Tasks.UniTask/WhenAnyLRPromise`1<UnityEngine.AudioClip>,Cysharp.Threading.Tasks.UniTask`1/Awaiter<UnityEngine.AudioClip>&) (at Assets/Plugins/UniTask/Runtime/UniTask.WhenAny.cs:120)
Cysharp.Threading.Tasks.UniTask/WhenAnyLRPromise`1/<>c<UnityEngine.AudioClip>:<.ctor>b__2_0 (object) (at Assets/Plugins/UniTask/Runtime/UniTask.WhenAny.cs:76)
Cysharp.Threading.Tasks.UniTaskCompletionSourceCore`1<UnityEngine.AudioClip>:TrySetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/UniTaskCompletionSource.cs:167)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`2<ChatdollKit.Extension.Azure.AzureTTSLoader/<DownloadAudioClipAsync>d__17, UnityEngine.AudioClip>:SetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:334)
ChatdollKit.Extension.Azure.AzureTTSLoader/<DownloadAudioClipAsync>d__17:MoveNext () (at Assets/ChatdollKit/Extension/Azure/AzureTTSLoader.cs:78)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`2<ChatdollKit.Extension.Azure.AzureTTSLoader/<DownloadAudioClipAsync>d__17, UnityEngine.AudioClip>:Run () (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:313)
Cysharp.Threading.Tasks.AwaiterActions:Continuation (object) (at Assets/Plugins/UniTask/Runtime/UniTask.cs:21)
Cysharp.Threading.Tasks.UniTaskCompletionSourceCore`1<UnityEngine.AudioClip>:TrySetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/UniTaskCompletionSource.cs:167)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`2<ChatdollKit.Extension.Azure.AzureTTSLoader/<DownloadAudioClipNativeAsync>d__18, UnityEngine.AudioClip>:SetException (System.Exception) (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:334)
ChatdollKit.Extension.Azure.AzureTTSLoader/<DownloadAudioClipNativeAsync>d__18:MoveNext () (at Assets/ChatdollKit/Extension/Azure/AzureTTSLoader.cs:109)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`2<ChatdollKit.Extension.Azure.AzureTTSLoader/<DownloadAudioClipNativeAsync>d__18, UnityEngine.AudioClip>:Run () (at Assets/Plugins/UniTask/Runtime/CompilerServices/StateMachineRunner.cs:313)
Cysharp.Threading.Tasks.Internal.PooledDelegate`1<UnityEngine.AsyncOperation>:Run (UnityEngine.AsyncOperation) (at Assets/Plugins/UniTask/Runtime/Internal/PooledDelegate.cs:46)
UnityEngine.AsyncOperation:InvokeCompletionEvent ()
uezo commented 1 year ago

Hi @nakazax , thank you for reporting an issue🥰 I couldn't notice this issue because it looks working well with ja-JP-HarukaRUS in my environment. I will change the out-of-the-box default speaker name in the next update.

Will you give me your personal recommendation for the speaker? (ja-JP, female)

nakazax commented 1 year ago

Hi @uezo, thanks for your comment. The Azure document says

you used a standard voice with your Speech resource prior to September 1, 2021 then you can continue to do so until August 31, 2024. All other Speech resources can only use prebuilt neural voices.

It is likely that you were unaffected by this if you had been utilizing a speech service that was created prior to July 2021.

My personal recommendation is ja-JP-NanamiNeural. It's the most standard female voice, with no noticeable quirks, making it easy to understand. You can listen to voice samples of this and other voices at the Voice Gallery.

nakazax commented 1 year ago

Hi @uezo, I created demo videos of ChatdollKit and Azure TTS neural voices, so I'll share the links with you.

After listening to the demos, I think the ja-JP-AoiNeural voice may match better with this 3D character. P.S. I created a C# script using Azure OpenAI Services' ChatGPT as an AI engine and used it. I'll send a Pull Request later separately from this issue. Thanks!

uezo commented 1 year ago

Hi @nakazax , Thank you so much! Okay, ja-JP-AoiNeural is the next OOTB default.