Analysis of botpress

[sojharo]

As discussed on slack, this task is to do the analysis of botpress, an open-source bot framework and solution. I would first read the documentation and website and would then do its setup on localhost to do the hands on testing.

[sojharo]

I have watched all the videos by botpress on youtube here:

https://www.youtube.com/channel/UCEHfE71jUmWbe_5DtbO3fIA/videos?sort=dd&view=0&shelf_id=0

I have looked into some documentation at high level and created the following draft document:

https://docs.google.com/document/d/13VMxK5h7fPFQGKGOZuZJTsS7-TG7raerksZcQOxwbV0/edit

I haven't completed the document yet. There are following items that I need to do in this task:

• Read the documentation in depth here
• Read about open source machine learning engine rasa.ai
• Run the project locally and build one sample bot, try integrating api.ai and messenger, github

[sojharo]

I have completed the BotPress document. Please go through the document and let me know your comments:

https://docs.google.com/document/d/13VMxK5h7fPFQGKGOZuZJTsS7-TG7raerksZcQOxwbV0/edit#

@dayemsiddiqui please must read.

Next, if we decide to go with the Botpress (use it fully, or use it to get UI code from them), then I would like to spend some time in understanding the code of BotPress. If we decide to use it fully then also as next step I would like to look into Rasa.AI, an open source machine learning engine.

[sojharo]

I have also implemented a sample bot, results of which are discussed in document.

[jekram]

@sojharo

1. I would like you to walk you thru this deck in the meeting today.
2. As I spoke to you yesterday please draw a diagram how it fits with API.AI and different channels. Take the current API.AI and show how it works and integrates with it.
3. What are the functions in the Dash Board that we have can be done with BotPress

[jekram]

@dayemsiddiqui @zarmeen92 Please read this.

[sojharo]

1. Yes.

2. Yes, I would draw the diagram now. In all cases it won't be different than the following diagrams that we already have made.

As stated in the document, Botpress doesn't provide us the UI to train the bot. The integration of botpress with API.AI shows the UI which let us insert the access token and then decide whether we want to do something with actionable data coming from api.ai or send the api.ai response as it is to the customer. I will show a small demo of botpress today.

3. Yes, I am adding these in document.

[sojharo]

Yesterday, we had more discussion on this and as discussed in the meeting, I have started creating a separate document which is under construction. I would also cater the questions in the above comment in it. Document would complete today.

[jekram]

@sojharo Please create it with the same three use case.

1. A stand alone ChatBot
2. ChatBot as an assistant
3. Message escalation from Chat Bot

The Channels we should focus on are:

1. Web
2. Messanger
3. Voice
4. SMS

The overall design should be the same as above in the picture. The big difference would be in the task list. So instead of coding we would implement and enable some services.

We should only implement what is absolutely required and we would enhance it later.

[sojharo]

Please see the new document. It is based on our previous document. I have highlighted the new things added in blue text. In the tasks list, I have highlighted the text in grey for the tasks that we would not do and they will be taken from botpress.

https://docs.google.com/document/d/1tIuSV_siFVCfBq8Qnw7zrtXFo5tftSNfuaJ5N_XhapQ/edit#

[sojharo]

We would use this new document for design of our project and I have excluded the comparison between API.AI and MS bot framework in current document. It is in the previous version.

[sojharo]

I went through the procedure of API.AI for Amazon Alexa. I found that there is basically no integration between API.AI and Amazon Alexa. API.AI is not using any webhook or Rest API of Amazon Alexa to use its services or give own services to Alexa. In the integrations page, when we click on Amazon Alexa, it exports some files for us that we can import in Alexa. After this, there is no connection between API.AI and Amazon Alexa.

Also, from the integration documentation, people are confused and asking for webhook or api place to do connection between API.AI and Amazon Alexa.

https://discuss.api.ai/t/integrating-with-alexa/111/2

The exported file contains the questions that we had given to API.AI for training. From the integration documentation, they are not doing any voice related work with Alexa. The exported questions are in format of text and then they are fed to Alexa.

https://api.ai/docs/integrations/alexa-exporter

Concluding, the API.AI is not doing any audio / voice thing with Alexa. So, I think we would have to it treat as integration and exclude it from voice section. The way we have discussed the voice input and output currently will work for our use cases. As far as other integrations are concerned which have their own audio processing, then they do this TTS and STT work on their own end and just send us the query.

We can only do voice input / output on our own platforms which are created by us i.e. web widget, android widget, and iOS widget. Because here we would have the freedom to show the microphone on the UI to let user give voice input. We can’t modify the UI of other chat platforms to include voice input if they currently are not supporting voice input. For example, if we integrate messenger, then it would not be possible for us to do voice input and voice output there because messenger’s UI natively doesn’t support this.

The methods that we have already discussed above for how we would do TTS and STT are capable enough to meet our use cases. All other platforms will use our API to send us query in text and get response in text.

As required, I have gone through how API.AI integrates with Amazon Alexa. Also, I exported the files for Alexa and inspected them. I haven't created an account on Amazon Alexa yet but I went through some of the documentation of Alexa. There we can create Alexa Skills which could then be used on Amazon Echo devices. The exported files from API.Ai are also used to create Skill in Alexa. The skill means that we can create more personalized experience on Alexa. Kindly, let me know if I need to look more into capabilities of Alexa.

https://developer.amazon.com/alexa-skills-kit

[jekram]

@sojharo

Good job let's discuss this in the meeting.

Q: Alexa is doing the speech to text translation and sending the text to us? correct

Q: Alexa is similar to Google Speech?

Q: If I have an Intent in API.AI. Say to Buy Pizza. This is not sent to Alexa. We get the text from Alexa and then API.AI applies the normal logic?

[sojharo]

Q: Alexa is doing the speech to text translation and sending the text to us? correct

From the integration of API.AI, they didn't do this. They are not sending anything between API.AI and Alexa. API.AI exports just some files which we integrate in Alexa. After this, there is no communication between Alexa and API.AI

Q: Alexa is similar to Google Speech?

Yes, it is similar to Google Assistant. Both these are smart home devices. They are placed in home to make a smart home. They listen to instructions and then do functions on home like switch off lights, turn on ACs.

Q: If I have an Intent in API.AI. Say to Buy Pizza. This is not sent to Alexa. We get the text from Alexa and then API.AI applies the normal logic?

Up till now, I have seen such communication use case between Alexa and API.AI. They are not connected using webhooks with each other so they don't communicate with each other. There are many complains regarding this on API.AI discussion forum here:

https://discuss.api.ai/t/integrating-with-alexa/111/2

I am thinking to look into Alexa official documentation today to understand more. People in the above picture are talking about webhooks but integration document of API.AI just let us know how to export some files and import them in Alexa.

[jekram]

@sojharo I have sent you few email. It should not be that difficult.

https://arstechnica.com/business/2017/04/amazon-opens-up-alexa-voice-and-text-tech-for-developers-to-make-new-chatbots/

https://developer.amazon.com/alexa-voice-service

[sojharo]

Yes, Sir.

[sojharo]

I have gone through the following links in order to read up on Alexa.

https://arstechnica.com/business/2017/04/amazon-opens-up-alexa-voice-and-text-tech-for-developers-to-make-new-chatbots/

https://developer.amazon.com/alexa-voice-service

https://developer.amazon.com/alexa-skills-kit

https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/overviews/steps-to-build-a-custom-skill

https://developer.amazon.com/alexa-skills-kit/tutorials/fact-skill-1

We would let the speech to text conversion happen on Alexa. We would have to provide the skill kit for Alexa. After providing the the skill kit, we would connect it using webhook.

There was other thing that I found in your first article. It is that we can integrate alexa voice and text into our web widget to convert text from speech and speech from text. However, both Google Speech API and Alexa have their charges, it would be business decision to choose any of them to enable our own web widget, android and iOS widgets to use TTS and STT.