search

Azure / azure-sdk-for-js

This repository is for active development of the Azure SDK for JavaScript (NodeJS & Browser). For consumers of the SDK we recommend visiting our public developer docs at https://docs.microsoft.com/javascript/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-js.
MIT License
2.06k stars 1.19k forks source link

Possible Bug in CustomQuestionAnswering.getAnswers method #24164

Closed PuliNath closed 1 year ago

PuliNath commented 1 year ago

Describe the bug the results answers from customQuestionAnswering.getanswers do not contain METADATA, instead the array is empty. whereas direct api call via curl provided from language studio ,does produce the result. To Reproduce Steps to reproduce the behavior:

  1. use the getAnswers method from CustomquestionAnswering class from botbuilder-ai 4.15

Expected behavior The metadata parameter returned from language Studio via CustomQuestionAnswering .getanswers , should have metadata values as trained in the language studio custom question answering.

Screenshots If applicable, add screenshots to help explain your problem.

Additional context Could this be due to reason that a dictionary is fetched from the actual/normal curl shared in the language studio portal , the exact same dictionary might have errored out , when converting to an array perhaps? because the result from getanswers method has empty array. adding a link to the library : https://learn.microsoft.com/en-us/dotnet/api/microsoft.bot.builder.ai.qna.customquestionanswering.getanswersasync?view=botbuilder-dotnet-stable

azure-sdk commented 1 year ago

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Storage:0.28149313,Azure.Core:0.07695083,azure-spring:0.07574897'

xirzec commented 1 year ago

@PuliNath which SDK are you using? Is it https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/cognitiveservices/cognitiveservices-qnamaker-runtime ? Can you share a code snippet to reproduce the behavior you are seeing?

PuliNath commented 1 year ago

We are using the botframework SDK , which is on the node.js runtime (https://github.com/microsoft/botframework-sdk) specifically ,https://learn.microsoft.com/en-us/javascript/api/botbuilder-ai/?view=botbuilder-ts-latest package.

const { CustomQuestionAnswering } = require("botbuilder-ai");

const qnaEndpoint = { knowledgeBaseId: projectname endpointKey: key, host: languagestudioHost, qnaServiceType:"Language" }; const qnaMakerOptions = { top: 5 }; const QnAService = new CustomQuestionAnswering(qnaEndpoint, qnaMakerOptions, null,undefined, true)

console.log(await QnAService.getAnswers(context)) //here context is the turncontext.

//the results are accuracte but the results do not contain metaData.

{ "answers": [ { "questions": [ "question 1?", "question 2.0" ], "answer": "rsome answerr...", "confidenceScore": 0.6391, "id": 29, "source": "source.xlsx", "metadata":[] -- ------------->empty??? ( this has key /values trained in portal)

PuliNath commented 1 year ago

whereas if we make use of old qnamaker class from botbuilder-ai we get the meta data , doing this since qnamaker would be not available from 2025.

xirzec commented 1 year ago

Thanks for the additional details @PuliNath!

Since the SDK you are using isn't maintained in this repo, could you re-file your issue in the botframework-sdk repo? https://github.com/microsoft/botframework-sdk/issues/new?assignees=&labels=needs-triage%2C+bug&template=bot-framework-bug.md&title=

PuliNath commented 1 year ago

Sure , i have re-filed , closing here . thanks