How to implement all properties and events of the Web Speech API

[AndreasTaime]

Hey James, hey guys, I'm currently working on my bachelor thesis developing an e-Learning App which should enable users to simulate oral exams. As a part of it I'd like to make use of some properties like the SpeechGrammar, SpeechRecognitionAlternative etc.. Is there a way to address the complete Web Speech API in React or is it just a limited component / framework you did build?

I hope this is worth opening an issue, best regards, Andreas

P.S.: I've already implemented your component successfully in my React project. Anything is working fine. Just need a hint how to use the rest.

[JamesBrill]

Hi Andreas,

I'm not familiar with the rest of the Web Speech API, though I'd like to keep the scope of this particular module limited to the SpeechRecognition interface to avoid it becoming too complex.

That said, it might not be too hard for you to combine this component with the other Web Speech interfaces. This React component injects the speech recognition object into your component as a property called recognition. I'm not sure what you have in mind, but you could do something with this object before mounting your component. e.g.

componentWillMount() {
  const { recognition } = this.props;
  const grammar = '#JSGF V1.0; grammar colors; public <color> = aqua | azure | beige | bisque | black | blue | brown | chocolate | coral | crimson | cyan | fuchsia | ghostwhite | gold | goldenrod | gray | green | indigo | ivory | khaki | lavender | lime | linen | magenta | maroon | moccasin | navy | olive | orange | orchid | peru | pink | plum | purple | red | salmon | sienna | silver | snow | tan | teal | thistle | tomato | turquoise | violet | white | yellow ;'
  const speechRecognitionList = new SpeechGrammarList();
  speechRecognitionList.addFromString(grammar, 1);
  recognition.grammars = speechRecognitionList;
}

Then the transcript will presumably be limited to words from the custom grammar you set up. This is just an example - I've not explored the rest of the Web Speech API. However, if I had to combine the API with this component in some way, I'd modify the recognition property in the componentWillMount lifecycle hook.

Best of luck with your bachelor thesis!

James

[AndreasTaime]

Thanks for the quick answer! Your example is kind of what I was hoping for. Unfortunately I encounter the same problem I did when trying beforehand. Trying to setup a constructor with the new-property throws following TypeError. I'm clearly missing something but am not able to initialize it correctly. Also tried to set it up in the constructor function - same error. Any idea? Thanks again, Andreas

[JamesBrill]

@AndreasTaime Each browser has its own implementation of Web Speech APIs. Chrome, for example, calls its implementation of Speech Grammar List webkitSpeechGrammarList. Usually, this implementation is accessible on the browser's window object.

To give you an idea of how to write something that'll work on most browsers, I had to do this to ensure that this module got a Speech Recognition object from most of the major browsers:

    const BrowserSpeechRecognition =
        window.SpeechRecognition ||
        window.webkitSpeechRecognition ||
        window.mozSpeechRecognition ||
        window.msSpeechRecognition ||
        window.oSpeechRecognition

BrowserSpeechRecognition was then a prototype I could instantiate with new BrowserSpeechRecognition(). If the user is on a browser that doesn't support Speech Recognition, BrowserSpeechRecognition would be undefined. In that case, I set browserSupportsSpeechRecognition to false so the consumer of the module can render some UI saying something like "sorry, this feature is not supported".

So you could try something like the following (note that I've not tried this myself):

    const SpeechGrammarList =
        window.SpeechGrammarList ||
        window.webkitSpeechGrammarList ||
        window.mozSpeechGrammarList ||
        window.msSpeechGrammarList ||
        window.oSpeechGrammarList;
    if (SpeechGrammarList) {
      const speechRecognitionList = new SpeechGrammarList();
      // Use speechRecognitionList
    } else {
      // SpeechGrammarList not supported by the user's browser; show this in the UI
    }