awslabs / aws-lex-browser-audio-capture

An example web application using the Lex JavaScript SDK to send and receive audio from the Lex PostContent API. Demonstrates how to capture an audio device, record audio, and convert the audio into a format that Lex will recognize, and play the response. All from a web browser.
MIT No Attribution
165 stars 74 forks source link

Force close (stop) a conversation #6

Closed shifatul-i closed 5 years ago

shifatul-i commented 6 years ago

When a conversation is in progress, how do i cancel (stop) it (without using voice)

Or when it is sending the voice to the server, I don't want to continue the conversation. how do I forcefully stop it.

I tried conversation.reset(); but it dosent work always, callback gets called.

My code:

function listen() {
    if (currentState === 'Passive') {
        updateState('Initializing');
        conversation = new LexAudio.conversation(config, function (state) {
            updateState(state);
        }, function (data) {
            console.log('Transcript: ', data.inputTranscript, ", Response: ", data.message);
        }, function (error) {
            message.textContent = error;
        }, function (timeDomain, bufferLength) {
            // visualize
        });
        conversation.advanceConversation();
    } else {
        conversation.reset();
        updateState('Passive');
    }
}
palafranchise commented 6 years ago

The first callback is invoked for each state change, you can add logic there to abort a conversation if you want to. Not sure if that's the what you're looking for though... where is the cancellation action/information coming from? Another option--you could use an external cancel button to reset the conversation object.

mridulsaran commented 6 years ago

I tried calling the "conversation.reset()" using a different button, still it is not terminating the flow.

devansvd commented 6 years ago

@palafranchise I tried the logic like you mentioned.

Here is the scenerio,

PASSIVE state at the beginning, when clicked state change to LISTENING Then to SENDING sending the audio and state becomes SPEAKING. Again it goes back to LISTENING state unless it receives a dialogstate as Failed or Fulfilled or ReadyForFulfillment to hits PASSIVE.

I want to terminate after one full cycle completed, not to started listening state again.

My only option is to check for SPEAKING state and do conversation.reset(); like you suggested. But this way it is not playing the last response received from Lex.

is there is anyway we can do that without changing the code directly ?

Temp Solution:

I directly changed the code in the aws-lex-audio.js to start for listening again and terminate it.

        var Speaking = function(state) {
            this.state = state;
            state.message = state.messages.SPEAKING;
            this.advanceConversation = function() {
                if (state.audioOutput.contentType === 'audio/mpeg') {
                    audioControl.play(state.audioOutput.audioStream, function() {
                        if (state.audioOutput.dialogState === 'ReadyForFulfillment' ||
                            state.audioOutput.dialogState === 'Fulfilled' ||
                            state.audioOutput.dialogState === 'Failed' ||
                            !state.config.silenceDetection) {
                            state.transition(new Initial(state));
                        } else {
                            //audioControl.startRecording(state.onSilence, state.onAudioData, state.config.silenceDetectionConfig);
                            state.transition(new Initial(state));
                            //state.transition(new Listening(state));
                        }
                    });
                } else {
                    state.transition(new Initial(state));
                }
            };
        };
palafranchise commented 5 years ago

Hi Vanthiyadevan,

Are you using a lambda function for Fulfillment or returning the parameters to the client? The library is setup to respond to the dialog state. So if the conversation is not in a terminal state it will continue. You can use a lambda function to control the state of the dialog. Enforce that there is just one full cycle for example.

Please re-open if this does not work. Happy to answer more questions.

Andrew

diosney commented 5 years ago

@palafranchise I've a similar need as @ThunderRoid.

Any chances that the conversation is forcefully interrupted manually from the client side?

That allows to add control button like enable/disable audio and so on.

diosney commented 5 years ago

Nevermind, I modified the script directly to access the audioControl from the conversation.

diosney commented 5 years ago

@ThunderRoid Also, I changed the flow to be initiated manually, instead of relaying on the silence algorithm, and that greatly improved the flow.

shifatul-i commented 5 years ago

@diosney could you share some code snippets. Thank you