Implement a quality check via the Dialogflow API

jayeshdesai4520 / chatbot-ng-internal

MIT License

3 stars 0 forks source link

Implement a quality check via the Dialogflow API #4

Open anbo-de opened 3 years ago

anbo-de commented 3 years ago

Context

The quality of a chatbot should be measured in multiple dimensions. The custom script should be written and test cases should be prepared for each dimension of quality check. Script should demonstrate quality check report after its execution.

Definition of Done (DoD)

[ ] Quality check for the intent recognition (Precision, Recall)
[ ] Quality check for the components recognition (Precision, Recall)
[ ] Quality check for KGQA system (in multiple languages)

jayeshdesai4520 commented 3 years ago

Hi, can I have context about this issue

RicardoUsbeck commented 3 years ago

We were discussing, that you need to test whether certain intents stay stable. That is, implement tests which fire not only one turn but multiple so you can see whether a conversation works or not.

jayeshdesai4520 commented 3 years ago

Hi, I know two platforms to test the chatbot 1) https://www.botium.ai/ 2) https://qbox.ai/GetStarted

Botium has a free plan to test

qbox gives only 5 free tests

I have used both of them just to test the current intents of our chatbot and both of them works fine

below are the images of testing

Botium - screenshot

qbox.ai - screenshot

anbo-de commented 3 years ago

@jayeshdesai4520 Where is the current quality check available/accessible?

jayeshdesai4520 commented 3 years ago

@anbo-de here is the screenshot of the test which I ran today Botium was taking so much time so I did use Qbox.ai https://i.imgur.com/H1SGatb.png

so the result is I need to add more training phrases in 6 intents according to Test

anbo-de commented 3 years ago

@jayeshdesai4520 Please create list of test cases as preparation.