Create a web based chatbot with voice input and output
In this code pattern we will create a web based chat bot, but the twist here is that we'll be using voice input and output. For the conversation dialog we'll of course be using Watson Assistant, but we'll also be using Watson Speech To Text to capture the user's voice, and lastly we'll use Watson Text To Speech to playback the chatbots response to the user. The web application itself is built on top of JQuery and Python Flask.
When the reader has completed this code pattern they will understand how to:
- Make a Watson Speech To Text call using a Web Socket Connection
- Make a Watson Text to Speech REST API call
- Send and receive messages to Watson Assistant using REST APIs
- Integrate Watson Speech To Text, Watson Text To Speech and Watson Assistant in a web app
Flow
- User selects the microphone option on the browser and speaks.
- The voice is passed on to Watson Speech To Text using a Web Socket connection.
- The text from Watson Speech to Text is extracted and sent as input to Watson Assistant.
- The response from Watson Assistant is passed onto Watson Text to Speech.
- The audio output is sent to the web application and played back to the user, while the UI also displays the same text.
Included components
- Watson Speech-to-Text: A service that converts human voice into written text.
- Watson Text-to-Speech: Converts written text into natural sounding audio in a variety of languages and voices.
- Watson Assistant: Create a chatbot with a program that conducts a conversation via auditory or textual methods.
Featured technologies
- Flask: Python is a programming language that lets you work more quickly and integrate your systems more effectively.
- jQuery: It is a cross-platform JavaScript library designed to simplify the client-side scripting of HTML.
Deployment options
Click on one of the options below for instructions on deploying the app.
 |
 |
 |
Sample output
Follow the script below for a sample path in an insurance domain, or configure your own dialog under Watson Assistant.
Voice Bot: Hello. How can I help you?
User Reply: I am trying to buy an insurance
Voice Bot: You have come to the right place We offer you a smart, affordable and convenient way to safeguard the future for you and your family. Can I ask you a couple of questions to come up with the best plan for you?
User Reply: ok sure
Voice Bot: Please tell me your Name followed by Date Of Birth and Annual Income
User Reply: This is Mary
Voice Bot: Please enter your Date of Birth
User Reply: 2nd June 1945
Voice Bot: Your annual income?
User Reply: 1200000 dollars
Voice Bot: Do you smoke?
User Reply: Nope I dont
Voice Bot: Please tell me your gender
User Reply: Female
Voice Bot: Enter a protection type
Select one among Basic Life Cover, Fixed Income Protection or Increasing Income Protection.
User Reply: Life Cover
Voice Bot: What premium payment frequency would you prefer? Monthly, Quarterly, Half Yearly or Yearly?
User Reply: Monthly
Voice Bot: What term would you want to go for? Enter a range between 10 years to 35 years.
User Reply: 8 years
Voice Bot: Please enter a term ranging between 10 years to 35 years.
User Reply: 12
Voice Bot: Your yearly premium for 12 years under the Basic Life Cover plan is 67767 dollars. Do you want to try any other options or go ahead with buying this?
User Reply: No What is Basic Life Cover?
Voice Bot: On death during the Policy term, Sum Assured as a lump sum is paid to the nominee and the policy terminates.
User Reply: ok Thanks
Voice Bot: Have a great day
Links
Learn more
- Artificial Intelligence Code Patterns: Enjoyed this Code Pattern? Check out our other AI Code Patterns.
- AI and Data Code Pattern Playlist: Bookmark our playlist with all of our Code Pattern videos
- With Watson: Want to take your Watson app to the next level? Looking to utilize Watson Brand assets? Join the With Watson program to leverage exclusive brand, marketing, and tech resources to amplify and accelerate your Watson embedded commercial solution.
License
This code pattern is licensed under the Apache Software License, Version 2. Separate third party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 (DCO) and the Apache Software License, Version 2.