[Unity] Voice Command UI System

[chenchen2015]

Description Given game developers usually have all the inputs mapped in their game for features like teleport, reload, etc., we shouldn't just override their key mapping for our recording control. Use voice command to interact with our plugin seems like a better choice but we need to test our hypothesis with game developers.

Basically, voice command feature can be broken down into speech-to-text service and text interpretation service. Since we only have easy instructions like start and stop recording, the main difficulty is to have a reliable and fast speech-to-text service. We will need to investigate the possibility of using a streaming speech-to-text API. Estimate or evaluate its performance and impact on the existing game. If not possible, we could use a non-streaming voice API.

Task List

• Plan A - Google Streaming Speech-To-Text API
  • Find any possible working Google Streaming Speech-To-Text service API in Unity. (Found one, but it took me some effort to make it work...)
  • Integrate Google Streaming STT API to ShareVR and test performance
  • Profile performance and evaluate if the API could fit into our MVP
• Plan B - Watson Speech-To-Text API
  • [x] First, check if it's possible to have a streaming STT service
  • [x] If there's streaming STT service in Watson Unity SDK, test its performance in its native scene.
  • [x] Integrate the plugin into ShareVR and evaluate fit and performance.

[chenchen2015]

Watson Speech-To-Text API has been integrated into SDK v0.2 release with the keyword detection system.

[chenchen2015]

Voice Command + UI ? We only have two voice commands right now. Will need to refine the system so we can have better control, possibly with the assist of an UI system.