search

awslabs / aws-lex-browser-audio-capture

An example web application using the Lex JavaScript SDK to send and receive audio from the Lex PostContent API. Demonstrates how to capture an audio device, record audio, and convert the audio into a format that Lex will recognize, and play the response. All from a web browser.
MIT No Attribution
165 stars 74 forks source link

Apparent duplication of spoken content after the first submission #7

Closed colingoldberg closed 6 years ago

colingoldberg commented 6 years ago

Hi Andrew,

As suggested, here is a copy of my posting in the discussion on the blog page.

In my recent testing I have noticed what appears to be a bug. I am not sure if this is in the aws-lex-audio.js file or in Lex.

If the problem is in the aws-lex-audio.js file - ie. that it does not appear to be initializing the recorder buffer in each cycle, it is a guess based on my seeing that Lex is passing uninitialized slot and transcript text to my Lambda function on subsequent tests.

So for example: I might say "give me the summary". "give" is (correctly) put in a slot named actionVerb. In the first cycle the Lambda function gets:

transcript = "give me the summary" actionVerb = "give"

If I click on the (mic) image and repeat "give me the summary", the Lambda function shows:

transcript = "give me the summary give me the summary" actionVerb = "give give"

Now this may be a Lex problem, so please let me know if my guess - that the recorder buffer is not being initialized each time - is wrong. I looked at the "startRecording" function and it looks ok.

I did originally take the code from github, and made only one addition for sessionAttributes (plus some console.logs).

I have just compared aws-lex-audio.js on github with what I have and this shows as: ` 231c231 < var DEFAULT_USER_ID = 'LexTester';

var DEFAULT_USER_ID = 'userId'; 353,354d346 < console.log('Sending:'); < console.log(state.lexConfig); 357d348 < //console.log(data); 402d392 < lexConfig.sessionAttributes = lexConfig.hasOwnProperty('sessionAttributes') ? lexConfig.sessionAttributes : {}; `

When I restart the client (in chrome), it does not show duplicates.

OS: macosx 10.13.4 Browser: chrome

Any help is appreciated.

Colin Goldberg

palafranchise commented 6 years ago

Thanks for posting this here! If you change your Intent to "Return parameters to client" do you see the same behavior?

palafranchise commented 6 years ago

Closing due to lack of activity. Please re-open if this is still an issue.

colingoldberg commented 6 years ago

Sorry, I missed your question of 15 days ago. Setting it to "Return parameters to client", I am not sure if I am looking at the best place to detect possible duplication. I had the javascript console open, and the response the second time was the same as the first time - so it does not seem as if there was duplication.

Should I look anywhere else to detect this?

Colin

colingoldberg commented 6 years ago

I have just tested again, and I think I have to retract my last conclusion.

I created a new intent, and have been testing it with my copy of your code on github. "Return parameters to client" is the (unchanged) setting.

I have evidence of 'duplication' (or 'not initializing the buffer...') - see below.

My utterance was "set the context to geography". My steps were:

The javascript console shows: image

Regards

Colin Goldberg

palafranchise commented 6 years ago

Okay, interesting. It seems that the conversation did not advance after you clicked "allow" the first time. Can you help me answer the following questions?

Assuming the above two answers, the behavior you should see is that Chrome prompts you for access to the mic only once. Once you say "yes" it should not ask you again (it saves your preference) and the conversation state should transition to Listening. Subsequent page loads should automatically grant you access to the mic without a popup. So, I'm curious why you are being prompted twice (or, more than once for that matter)... are you always prompted or only after clearing your preferences?

colingoldberg commented 6 years ago

Testing this again...

Yes, I am using Chrome. I hope the above sequence shows that Chrome is asking me to allow access when it shows "Passive" - ie. each time. I was not clearing anything during the test.

Also, it is silent for each response - no audio output accompanying the receipt of the responses above.

palafranchise commented 6 years ago

Interesting, I played around with Chrome and the only way I was able to get it to prompt me every time was to open the file locally (rather than serve it locally). Presumable Chrome does not store preferences for local file paths. In any case, I cannot reproduce the issue you're seeing--when prompted by Chrome and after allowing access the example code transitions to listening. Perhaps we can get together on a conference call to debug? Is it okay if I contact you at your github account email address to setup a call?

colingoldberg commented 6 years ago

I have been testing using a local file - maybe that's the answer - or part of it.

I will test it from my website to make sure - give me a little time as I am refactoring the site. I will be happy to communicate by email shortly to set up a call if necessary.

palafranchise commented 6 years ago

Sounds good!

On Wed, May 23, 2018, 6:42 AM Colin Goldberg notifications@github.com wrote:

I have been testing using a local file - maybe that's the answer - or part of it.

I will test it from my website to make sure - give me a little time as I am refactoring the site. I will be happy to communicate by email shortly to set up a call if necessary.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/awslabs/aws-lex-browser-audio-capture/issues/7#issuecomment-391351028, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOWVFRx7tgfmPao4OPbHRMIif5iMIIzks5t1Wc8gaJpZM4TuCJn .

colingoldberg commented 6 years ago

If you want to schedule a time for next week, please let me know your preferences (day/time/zone) - email is ok.

palafranchise commented 6 years ago

Looks like the buffer was not cleared when there was no audio response. Made a change to ensure the buffer is cleared after every interaction--even if there's no response from Lex.