search

microsoft / BotBuilder-Samples

Welcome to the Bot Framework samples repository. Here you will find task-focused samples in C#, JavaScript/TypeScript, and Python to help you get started with the Bot Framework SDK!
https://github.com/Microsoft/botframework
MIT License
4.37k stars 4.87k forks source link

Issue regarding active learning and questions with prompts(Custom question and answering [CQA]) #3878

Open charith-phamax opened 1 year ago

charith-phamax commented 1 year ago

Github issues for C# /JS / Java/ Python should be used for bugs and feature requests. Use Stack Overflow for general "how-to" questions.

Sample information

  1. Sample type: 48.customQABot-all-features
  2. Sample language: nodejs
  3. Sample name: Custom Q&A Bot with all features

    Describe the bug

    When using active learning, if a question is asked which is unclear, we get options "Did you mean? [question1][question2]" etc. If the selected question contains prompts, then on selecting anyone of those prompts, we are encountering an issue where the bot is not correctly identifying user intents and providing relevant responses. If the question selected from active learning card("did you mean" card), is asked directly and not from the card, then response is relevant.

To Reproduce

Steps to reproduce the behavior:

  1. Clone the 48.customQABot-all-features sample
  2. Keep active learning feature as TRUE for your "custom question and answering" project
  3. Test the bot by asking questions which are unclear.
  4. On getting "Did you mean? [question1][question2]" choose question which has prompts
  5. Then select one of the prompt and evaluate the result.
  6. To get correct response, directly ask the question without using the active learning card. then select the same prompt and evaluate the response.

Expected behavior

We expect the custom Q&A bot to correctly identify user questions and provide relevant responses based on the active learning and prompt features.

Screenshots

  1. This screenshot contains active learning card. image
  2. This screenshot contains the prompts for the selected question. image
  3. This screenshot contains response to the selected prompt.(not relevant response) image
  4. This screenshot contains the question (asked directly). image
  5. This screenshot contains the response to the prompt selected. image

Additional context

No additional context

Amrit10737726 commented 6 months ago

It seems there's uncertainty about the relevance of tsv and how it triggers certain actions within the bot. plan is to reach out to the customer for clarification on their tsv and aims to revisit the original issue once more information is available. The complexity of the problem is acknowledged, but efforts are being made to address it effectively.