search

microsoft / MixedRealityToolkit-Unity

This repository is for the legacy Mixed Reality Toolkit (MRTK) v2. For the latest version of the MRTK please visit https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
https://aka.ms/mrtkdocs
MIT License
6k stars 2.12k forks source link

Need an easy way to do Text to Speech #25

Closed jbienzms closed 8 years ago

jbienzms commented 8 years ago

Windows 10 supports text to speech via SpeechSynthesizer but this isn't easily used from Unity because of the complexities in converting from SpeechSynthesisStream to AudioClip. I propose a new component called TextToSpeechManager (which I've already written) to do this work for us.

jbienzms commented 8 years ago

Oh, I forgot that I'm supposed to describe how I plan to solve this problem. I have been able to convert SpeechSyntehsisStream to a byte[] containing WAV data. I then found code to load WAV data into a mono Unity AudioClip. From there, we can assign that audio clip to an Audio Source which allows the voice to be positioned in 3D space. The TextToSpeechManager component does all of this work. The only things the developer needs to do are:

  1. Create an AudioSource somewhere in the scene. I recommend creating it on an empty GameObject that is a child of the Main Camera and is positioned about 0.6 units above the camera.
  2. Add the TextToSpeechManager component to the scene.
  3. Assoicate the AudioSource with the manager.
  4. In code call TextToSpeechManager.Instance.Speak("Hello World");

The component takes care of generating the speech using SpeechSysntesizer, converting the output stream to WAV data then on to an AudioClip. It sets the AudioSource.clip property and then calls AudioSource.play. This all happens in about 500ms.

keveleigh commented 8 years ago

I agree! I've been working on something like this recently as well, but have been stuck at the "convert the WAV data to an AudioClip" step. Sounds like you've solved that, so I think this would be a welcome addition. Feel free to create a pull request for review.

jbienzms commented 8 years ago

Awesome! Pull request has been submitted. You can review it here:

https://github.com/Microsoft/HoloToolkit-Unity/pull/26

jbienzms commented 8 years ago

Updated to new pull request:

https://github.com/Microsoft/HoloToolkit-Unity/pull/38

jbienzms commented 8 years ago

Updated to a third pull request:

https://github.com/Microsoft/HoloToolkit-Unity/pull/48

jwittner commented 8 years ago

Awesome! Left some comments. Out of curiosity, why the new Pull Request every time you make changes? If you update your branch the currently standing PR will update to reflect that merge instead.

Can you assign this issue to yourself? That way it's clear from the outside that you're working on it. =)

jbienzms commented 8 years ago

As for the new pull request, well, to be honest I haven't had a pull request rejected before. The one's I've done were accepted first time so I never had to go back and rework a submission. Good to know it works that way, thank you.

I would be happy to assign the issue to myself, but when I click the Edit button I'm only able to change the title. I don't think I have the permissions to assign the issue. Can you help?

keveleigh commented 8 years ago

I'm not sure where the Assignees list is filled from, but I wasn't able to find you in the list for some reason.

kenyee commented 8 years ago

+1 used it this weekend at the Boston hackathon and it worked very well. Thanks, Jared!

mobrien650 commented 8 years ago

Used at the Boston MS Hololens Hackathon this weekend and it worked perfect for what we needed.

mattThousand commented 8 years ago

Another +1, this would be great feature to bring in

NeerajW commented 8 years ago

@jbienzms, have you added a test scene to your pull? Please check out the Contributing.md’s implementation section.

Once your test scene is added and readme updated, I’d be happy to merge.

jwittner commented 8 years ago

This was fixed by the merge of #48 correct?

jbienzms commented 8 years ago

Correct @jwittner. Sorry I forgot to close the issue. Closing now.