Speech Recognition Continuos mode

[Danyzzz]

Hi, is it possible a Speech Recognition Continuos?

Thanks in advance!

[pbakondy]

It is possible on iOS

[Danyzzz]

I'm interested on Android... Like a "ok Google"

[sdkcarlos]

@Danyzzz The speech recognizer doesn't offer such a continuous mode. However i've achieved to make a "continuous mode" in the same way that everybody does (and with acceptable results) with the Speech Recognition API in the browser : once the speech finishes, restart it again.

However i've faced the 2 obvious problems:

1. The success callback is executed only once (read how to solve it here).
2. As we need to use the same success callback to send results, we need to filter someway with Javascript when the result is sent from Java to Javascript (to know wheter the plugins says that the speech recognition ends or the results were retrieven). We can simply send an object from Java to JS with the structure:

{
    "event":"event-name",
    // Only for the onResult event
    "matches": ["Recognized text","Recognized text","Recognized text"]
}

Then, the RecognitionListener events of the Java code should be modified too as follows (returns the previously mentioned structure):

@Override
public void onEndOfSpeech() {
    try {
        JSONObject response = new JSONObject();
        response.put("event", "speech.onend");
        callbackContext.success(response.toString());
    } catch (Exception e) {
        e.printStackTrace();
        callbackContext.error(e.getMessage());
    }
}

@Override
public void onResults(Bundle results) {
    ArrayList<String> matches = results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
    Log.d(LOG_TAG, "SpeechRecognitionListener results: " + matches);

    try {
        JSONObject response = new JSONObject();
        // Send type of event "on result"
        response.put("event", "speech.onresults");
        // Send recognized text
        response.put("matches", new JSONArray(matches))
        callbackContext.success(response.toString());
    } catch (Exception e) {
        e.printStackTrace();
        callbackContext.error(e.getMessage());
    }
}

And with Javascript (speechRecognition.js) would be now:

startListening: function (successCallback, errorCallback, options) {
    options = options || {};
    cordova.exec(function (data) {
        var response = JSON.parse(data);

        // Note that you need to add some way of conditional to know wheter the continuous mode is used or the simple mode
        if (response.event == "speech.onend") {
            // Note that you need to wait 500ms, otherwise the error "recognition busy" will be triggered.
            setTimeout(function () {
                // Restart the recognition with the callbacks 
                window.plugins.speechRecognition.startListening(successCallback, errorCallback, options);
            }, 500);
        } else if (response.event == "speech.onresults") {
            console.log("Hey something was recognized, check out : ", response.matches);
        }

    }, errorCallback, 'SpeechRecognition', 'startListening', [options.language, options.matches, options.prompt, options.showPartial, options.showPopup]);
},

However i won't make a pull request because that would modify the entire structure in which the plugins works, i just let the idea here in case that someone really needs to make it work in continuous mode. Note that you need figure out how to set a "continuous option" to know when to start the plugin in continuous mode or normal mode.

[lsim6580]

looking at the google docs it looks like they support continuous speech, will this get implemented?

[renanbrenovital]

@sdkcarlos Thank you very much, I really needed to leave in continuous mode. I need some help to implement. I have modified the code but it is giving the following error:

..\pbakondy\S
peechRecognition.java:279: error: cannot find symbol
            JSONObject response = new JSONObject();
            ^

[Danyzzz]

There are news for use continuos mode?

[ragcsalo]

also how does it work on iOS?

[hiteshtare]

I need to develop an app where speech recognition will be continous in order to detect hotword for both iOS and Android platforms , Does Speech Recognition Continuos mode is supported in Ionic v5?

[dineshd1998]

@hiteshtare Bro did you get any solution for this bro?

[JulienLecoq]

I was also looking to continuous speech recognition, unfortunately it's still not supported and consumes a lot of power to restart speech recognition every time (see: https://developer.apple.com/documentation/speech/sfspeechrecognizer with the part on Create a Great User Experience for Speech Recognition).

I will try the wake word service of Porcupine : https://picovoice.ai/docs/porcupine/. The drawback is that it's a paid service if you have more than 3 users that use the service.