jsquire
commented
2 years ago Thank you for your feedback. Tagging and routing to the team member best able to assist.
Closed hebaelfadly closed 2 years ago
jsquire
commented
2 years ago Thank you for your feedback. Tagging and routing to the team member best able to assist.
ghost
commented
2 years ago Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @bingisbestest, @nerajput1607.
| Author: | hebaelfadly |
|---|---|
| Assignees: | heaths |
| Labels: | `Service Attention`, `customer-reported`, `question`, `Cognitive - QnA Maker` |
| Milestone: | - |
heaths
commented
2 years ago @rokulka this appear to be an issue with the QnA portal itself.
sahithikkss-zz
commented
2 years ago @hebaelfadly This appears more like a fallback answer when the KB did not have any answer for user question. "يرجى الدخول على الموقع الالكتروني لموؤسسة الامارات للخدمات الصحية https://www.ehs.gov.ae/ar/Pages/default.aspx"
A few things to check if my assumption above is correct.
Case 1: Abu Dhabi and Dubai words in user question are present in the actual answer text. So, it is possible that QnA Maker brought it with considerable confidence score - possibly > 50 (you can check via Inspect on Test pane)
Case 2: It is possible that none of QnAs in KB matched any word in the user question, hence "No good match found in KB." is the answer.
On www.qnamaker.ai there is no filtering based on score. Hence top 3 answers (Inspect and view) or default answer will be shown.
Instead of having the fallback answer as QnA in the KB, you can programmatically (via a Bot) show this answer whenever the top answer's score is not above a threshold.
Read more about confidence score Create bot - https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/tutorials/create-faq-bot-with-azure-bot-service?tabs=v1 Default threshold is 30% (0.3) i.e, only answers with > 30 score will be shown to bot user, otherwise bot's default answer is shown. QnAMakerOptions can be modified to provide a different threshold value.
Samples for your reference C# https://github.com/microsoft/BotBuilder-Samples/tree/main/samples/csharp_dotnetcore/49.qnamaker-all-features NodeJS https://github.com/microsoft/BotBuilder-Samples/tree/main/samples/javascript_nodejs/49.qnamaker-all-features
In both cases 1 and 2, configuring a higher bot score threshold and setting bot default answer to fallback answer can address the scenario. For any specific questions/concerns about improving the KB visit Tutorials on https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker.
A note about alternate questions, if alternate questions are of different concepts, they are better as two different QnAs.
In other words, alternate questions should be related to one concept to increase the chances of matching user's question phrases to a given QnA.
@jsquire @heaths This appears more of a user scenario and better KB organization challenge than a bug. Hope above diagnosis and suggestions help @hebaelfadly .
hebaelfadly
commented
2 years ago @sahithikkss thanks a lot for your response.
No, it's not the fallback answer, I have not set any fallback answer and the score for the first question is 26.94. Both questions are found just as is in the KB, even so the corresponding answer is not returned.
Regarding the alternate questions, I have uploaded the excel sheet as separate questions, but the default behavior of QnAMaker import feature is that if multiple questions have the same answer it creates only one question with alternatives, so it's not a scenario that I can handle from my side. Please tell me if I am wrong here or if there is an option to import them as separate questions.
sahithikkss-zz
commented
2 years ago @hebaelfadly Add a placeholder metadata which is unique to the row in the excel. Sample format.
Describe the bug I am trying to upload a QnA pairs excel file to through https://www.qnamaker.ai/ or through the API no difference. The file is uploaded normally, but when I try to test it through the test panel same questions are answered normally, some return wrong answer, and the others returns No good match found in KB.
For example:
Question: "أنا أعمل في أبوظبي والخطيبة من دبي فكيف العمل؟" Expected Answer: "يرجى الدخول على الموقع الالكتروني لموؤسسة الامارات للخدمات الصحية https://www.ehs.gov.ae/ar/Pages/default.aspx" Actual Answer: "في حالة التحويل من هيئة الصحة في دبي أو أبوظبي يجب أن يكون الترخيص ساري المفعول، وإذا لم يُجدد أو تم إلغائه يجب ألا يتجاوز فترة 6 شهور، وفي حالة تجاوز 6 شهور بيوم واحد لا يتم التحويل ويخضع المتقدم للامتحانات المقررة" which is the answer of a completely different question.
Question: "كم عدد الزيارات التي يجب القيام بها لعلاج عصب الأسنان؟" Expected Answer: "يرجى الدخول على الموقع الالكتروني لموؤسسة الامارات للخدمات الصحية https://www.ehs.gov.ae/ar/Pages/default.aspx" Actual Answer: No good match found in KB.
I have tried various work around to fix the issue and finally figured out that the error happens when there are alternative questions for the same answer. When I added QnAs manually and separate them, one question - one answer it works fine.
Expected behavior Return the question answer from the file.
Actual behavior (include Exception or Stack Trace) Return an answer of a completely different question.
To Reproduce Steps to reproduce the behavior (include a code snippet, screenshot, or any additional information that might help us reproduce the issue)
Environment: