An error occurs when a character tries to speak on WebGL

[moriya2535]

Hello,

I tried building ChatdollKit into WebGL based on this page (Usage 1~3). https://github.com/uezo/uLipSyncWebGL

The build was successful and ChatdollKit started on chrome (browser). When I spoke with Wake word, the character tried to respond, but it makes a buzzing noise and no text is displayed. ( It seems that voice recognition is successful as it does not respond to anything other than the Wake word.)

After that, Listening... was displayed. I tried to talk to the character, but it crashed with an error before the character spoke. ( It's crashing in framework.js)

It seems to crash when the character speaks. (something wrong with the speaker or GPT connection? )

I have confirmed that ChatdollKitDemo works correctly on Unity.

---

I said "It is good weather". Chatdollkit can also correctly recognize my speech. However, It is sending a chunk to Gemini and it seems to be crashing with an error that there is no API key. I have registered an Open AI key. If I register the Gemini key, will it solve the problem?

Any help is appreciated.

[uezo]

Hi @moriya2535 , If you are using GeminiServiceWebGL, the API key issued at Google AI Studio is required.

[moriya2535]

Hi @uezo Thank you for your reply. When I set the Gemini key in addition to the OpenAI key, it no longer stopped due to an error. And I changed the code below in OpenAITTSLoader.cs ,because it couldn't play mp3.

{ "response_format", "wav" },  //from mp3

And I changed the code below in OpenAITTSLoader.cs ,because sound was so slow that I couldn't hear it.

return AudioConverter.PCMToAudioClip(resp.Data, 1, 24000);  //from 8000

Now, I can hear the character's voice correctly. ( If there is a problem with my correction method, please let me know.)

I am currently working on resolving an issue where Japanese characters are not displayed as text. I think the following is the solution, but I'm having trouble figuring out how to go about it. Do you have any hints?

If you want to show multibyte characters in message window put the font that includes multibyte characters to your project and set it to message windows.

[uezo]

Hi @moriya2535 , Use the font that includes Japanese like IBM Plex Sans JP, Noto Sans or something like that.

[moriya2535]

Thank you for your quick reply. I will try based on your answer. thank you

[moriya2535]

Hi @uezo, Thanks to you, I can display Japanese text. Thank you for your kind support.