microsoft / cognitive-services-speech-sdk-js

Microsoft Azure Cognitive Services Speech SDK for JavaScript
Other
252 stars 91 forks source link

[Bug]: SSML bookmark tag returning junk value with a specifc voice #828

Open branaway opened 4 weeks ago

branaway commented 4 weeks ago

What happened?

for

<speak version='1.0' xmlns="http://www.w3.org/2001/10/synthesis" xmlns:mstts="https://www.w3.org/2001/mstts" xml:lang='en-US'>
        <voice name='zh-CN-XiaoxiaoMultilingualNeural'>
anything <bookmark mark="hello"/>  anything
        </voice>
      </speak>

The bookmark event returned weird characters: 배萓罵

Other models worked fine, such as: zh-CN-XiaoyuMultilingualNeural.

How does the bookmark have anything to do with the voice....

Version

1.36.0 (Latest)

What browser/platform are you seeing the problem on?

Safari

Relevant log output

No response

branaway commented 4 weeks ago

I was using the East Asia region, FYI

BrianMouncer commented 4 weeks ago

@branaway do you see this happen with the none multilingual voices like zh-CN-XiaorouNeural?

@yulin-li Do you know of any open issue around book mark events an specific voices, other than the inherint limitation of the OpenAI whisper voice? I would expect any of our neural voices to work, but I'm not sure about the multilingual voices.

branaway commented 4 weeks ago

zh-CN-XiaorouNeural

no issue with that voice

BrianMouncer commented 4 weeks ago

zh-CN-XiaorouNeural

no issue with that voice

ok, poor data sample size, but that supports the idea that the multilingual voices may have an issue with the bookmark label. I will follow up with our text to speech team, and see if they are aware of this.

In the meantime, you might try changing the bookmark label from hello, to either a number or some Chinese text that the multilingual engine would be less likely to mis-identify the source language of...

or

branaway commented 4 weeks ago

tried all various bookmarks in all kinds of symbols, languages. Did not work.

ForrestGumb commented 4 weeks ago

I can reproduce this issue. Will investigate.