fast loop of listening and sending

[MobileSuitTech]

Hi. I am using this solution to call the example Amazon Lex bot: OrderFlowers. After two interactions or three or four (it is random), the Sending and Listening functions fire up ramdomly and the voice feedback from the bot also fires up randomly. Sometimes a fast loop creates and alternates the bot saying one of the things it is supposed to say, overlapping itself.

[palafranchise]

Hi there. Can you tell me more about what you mean when you say fast loop? Also, please let me know what Browser, Browser version, and OS you are using.

A couple of possibilities: 1) The "Start conversation" button advances the conversation state without any checks. So if it's listening and waiting to detect silence and you advance the conversation state two requests will be made. One because you manually advanced the state and one when silence is detected. I've made a change to only advance the state manually when transitioning from the initial state which should prevent this behavior. 2) The example is pretty simplistic--it only transitions back to the initial state when playback is complete and the dialog state is 'ReadyForFulfillment'. For example if you are using a Lambda function and the state is 'Fulfilled' or you get a failure and the state is 'Failed' it will transition back to the listening state.

Please do send any additional information about your issue if the above items are unrelated or don't address it.

[MobileSuitTech]

Hello and thank you for your prompt response.

By fast loop I meant that effect that happened feels as if the code was a loop that loops a few times and starts the same voice action at each iteration. I am using Firefox 56 and Microsoft Edge on Windows 10.

Now the app listens for a long time after the user is done talking, even if the user clicks on the icon right after they are done talking. Then after a while the icon says "sending" and the app gets stuck there . This happens most of the time. I've gotten it to actually work and be responsive one time, out of about 10 tries. Sometimes I can get to the 4th question in my bot's string of questions.

[palafranchise]

Hi again,

This code has only been tested in Chrome and Firefox--this is mentioned in the companion blog post (perhaps not prominently enough :-)): https://aws.amazon.com/blogs/ai/capturing-voice-input-in-a-browser/ "The getUserMedia method is supported in Chrome, Firefox, Edge, and Opera. We tested the example code in Chrome and Firefox."

I successfully tested this morning with Firefox 56.0 (64-bit) on Windows 10 against the OrderFlowers bot. A couple of things to note:

• As of the last commit the example code only responds to clicks when in the Passive state (to start the conversation).
• It uses a very simple "silence detection" implementation (as an example of using AnalyserNode.getByteTimeDomainData) to stop recording and transition to sending. If you are in a noisy environment it may not detect silence immediately. From the comments: "Checks the time domain data to see if the amplitude of the sound waveform is more than 0.01 or less than -0.01. If it is, "noise" has been detected and it resets the start time. If the elapsed time reaches 1.5 seconds the silence callback is called."
• You can modify the silence detection behavior to suit your needs (adjusting the thresholds or adding your own silence detection function) or pull it out altogether and create a "click to send" experience. https://github.com/awslabs/aws-lex-browser-audio-capture/blob/master/scripts/recorder.js#L91

Happy hacking, let me know if you have any more questions/issues.

Cheers,

Andrew