search

aws-solutions / qnabot-on-aws

AWS QnABot is a multi-channel, multi-language conversational interface (chatbot) that responds to your customer's questions, answers, and feedback. The solution allows you to deploy a fully functional chatbot across multiple channels including chat, voice, SMS and Amazon Alexa.
https://aws.amazon.com/solutions/implementations/aws-qnabot
Apache License 2.0
401 stars 253 forks source link

Voice recorder button throws an exception when qnabot is embedded on website #693

Closed chanbi-aws closed 9 months ago

chanbi-aws commented 9 months ago

Describe the bug Selecting the mic button when the qnabot is embedded in a HTML website (parent.html) results in an exception: "Uncaught (in promise) DOMException: The element has no supported sources." The error is on line 29 of lex-web-ui.min.js, as part of setAudioAutoPlay.

To Reproduce Deploy the bot as an embedded UI and click the mic button.

Expected behavior Launch the audio recorder associated with the browser.

Please complete the following information about the solution:

To get the version of the solution, you can look at the description of the created CloudFormation stack. For example, "(SO0189) QnABot [...] v5.4.5".

Screenshots If applicable, add screenshots to help explain your problem (please DO NOT include sensitive information).

Additional context Add any other context about the problem here.

dougtoppin commented 9 months ago

@chanbi-aws thanks for your report, we will investigate and get back to you

michaelin-96 commented 9 months ago

Hi @chanbi-aws, I assume you are using v5.4.5 from your latest post. I recommend trying to upgrade to the latest version v5.5.0. I wasn't able to replicate this issue when deploying qnabot v5.5.0. Let me know if you still come across issues! Thanks.

michaelin-96 commented 9 months ago

After discussion with @chanbi-aws, this seems to be a bug from lex-web-ui, which our QnABot solution does not manage.

chanbi-aws commented 9 months ago

This is a different issue to the 'invalid ssml'.

bobpskier commented 9 months ago

@michaelin-96 @chanbi-aws If the LLM (Bedrock) response is generating output to be used in a voice based response through QnABot, the handlers managing the request/response with Bedrock will need to manage output in a voice based SSML manner. That is, the Bedrock integration will need to provide a voice based response in valid SSML if that is supported. LexWebUi is not going to manage output from the Lex bot and perform correction to account for incorrectly formatted SSML. The integration in QnABot needs to identify that the request was made in voice mode and that the output should be in voice mode using valid SSML.

Is the response payload being provided to LexWebUi when the error occurs available to investigate further?

Is there a public endpoint that can be exercised to investigate?

chanbi-aws commented 9 months ago

The issue occurs prior to any request to the LLM. The error is thrown when you click the 'mic' icon to record your voice input. Maybe this is an issue that needs to be raised with https://github.com/aws-samples/aws-lex-web-ui.

bobpskier commented 9 months ago

@chanbi-aws Is the embedded version of LexWebUi being used or was the full version from https://github.com/aws-samples/aws-lex-web-ui installed? Sounds like it is the full version based on the use of parent.html but want to confirm. If its the full version then this should be submitted against LexWebUi. Note a number of problems with voice mode were resolved in LexWebUi 0.20.2 which was released on 2023-11-28. So first step is going to be to upgrade to the latest LexWebUi at least greater than or equal to 0.20.2.

michaelin-96 commented 9 months ago

@bobpskier This comment should refer to this ticket 694. In terms of the mic issue in the embedded page, I recommend using the latest lex build (20.6), otherwise this may be a bug for lex-web-ui. Closing this ticket out now

@michaelin-96 @chanbi-aws If the LLM (Bedrock) response is generating output to be used in a voice based response through QnABot, the handlers managing the request/response with Bedrock will need to manage output in a voice based SSML manner. That is, the Bedrock integration will need to provide a voice based response in valid SSML if that is supported. LexWebUi is not going to manage output from the Lex bot and perform correction to account for incorrectly formatted SSML. The integration in QnABot needs to identify that the request was made in voice mode and that the output should be in voice mode using valid SSML.

Is the response payload being provided to LexWebUi when the error occurs available to investigate further?

Is there a public endpoint that can be exercised to investigate?

chanbi-aws commented 9 months ago

I will raise an issue with aws-lex-web-ui. Updating the stack with the master.yaml (https://github.com/aws-samples/aws-lex-web-ui/blob/master/templates/master.yaml) successfully upgrades to v0.20.6, but when the UI initialises with version 0.20.5. Browser console output: "lex-web-ui.min.js:29 successfully initialized lex web ui version: 0.20.5"