search

microsoft / BotFramework-Composer

Dialog creation and management for Microsoft Bot Framework Applications
https://docs.microsoft.com/en-us/composer/
MIT License
868 stars 372 forks source link

Bot Orchestrator : Unable to achieve disambiguation of common utterances present across multiple PVA skills #9581

Closed pckyn closed 1 year ago

pckyn commented 1 year ago

Describe the bug

We have added multiple PVA skills in Azure Bot (C#) and enabled orchestrator. The unique utterances are getting navigated to associated PVA skill and getting the appropriate response.

Now, we have common utterances in multiple PVA skills .. for an example PVA Skill 1 : utterances - Create incident PVA Skill 2: utterances - Create incident for HR Functions PVA Skill 3: utterances - Create incident for IT functions

Now, when we search Create incident , it goes into unknown intent. and unable to proceed further.

We tried another approach to solve this issue .. there is an trigger phrase - "Duplicated intents recognized"

Using this Azure Bot goes into disambiguation and gives an option to use to select which Skill you want to go for utterances asked "Create Incident" .. when we select appropriate skill and submit ... the bot doesn't respond.. found that utterance is not getting passed to the selected skill..

Version

2.1.2

Browser

OS

To Reproduce

Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior

When we select the PVA skill and submit .. it should send the asked utterance to the selected PVA skill and respond appropriately.

Screenshots

image image

Additional context

InfinytRam commented 1 year ago

Thanks @pckyn, I'm investigating.

InfinytRam commented 1 year ago

Hi @pckyn, could you please provide the steps to reproduce this issue?

pckyn commented 1 year ago

Hi @ramfattah - The steps are as below

  1. Create an Azure Bot
  2. Create PVA bots and add few similar utterances in all the PVA bots
  3. Add PVA bot skills in azure bot, may be please add two to three PVA skills to test the usecase
  4. Enable Orchestrator in azure bot
  5. Publish the azure bot in azure
  6. Search similar utterances that are available in skills and observe the response

Please let me know if any additional details are needed.

InfinytRam commented 1 year ago

Thanks for your patience @pckyn, we are actively investigating.

InfinytRam commented 1 year ago

Hi @pckyn,

I tried to isolate this disambiguation of common utternaces issue between PVA and Composer.
It appears I'm not able to reproduce this issue using standalone Composer bot (.NET) and Composer skills. (Not PVA skills)

The Duplicated intents recognized works as expected for Composer Skills.

Perhaps this issue arises only when PVA skills with similar utterances is introduced to the picture.

Sharing the steps I took:

  1. Created Root Bot in Composer (C# .NET Bot)
  2. Added 3 Skills to the Root bot, by following this documentation
    1. Skill 1
      1. Skill Name: CreateIncidentSkill
      2. Trigger phrase in Root bot: Create incident
    2. Skill 2
      1. Skill Name: CreateHRIncidentSkill
      2. Trigger phrase in Root bot: Create incident for HR Functions
    3. Skill 3
      1. Skill Name: CreateITIncidentSkill
      2. Trigger phrase in Root bot: Create incident for IT functions
  3. In Root Bot, add Duplicated intents recognized trigger
  4. Run the bot
  5. Type one of the skill trigger phrases to invoke the appropriate skill, example Create incident for HR Functions
  6. Duplicated intents recognized will be triggered
  7. Select your skill in the card
  8. The correct skill is invoked through Duplicated intents recognized trigger

Video Demo:

https://github.com/microsoft/BotFramework-Composer/assets/38049078/27b18ec5-4b04-4c16-9a88-0ec9765833bd

Attached is .zip Composer bot demo: https://drive.google.com/file/d/19GeN1HBxjS_n7z3Nnn4t9D8T1SOR0r-i/view?usp=drive_link

Again, I used Composer bot skills (not PVA skills) to try to isolate this issue from PVA.

InfinytRam commented 1 year ago

For next steps, I will add PVA skills to see if the issue is reproducible.

InfinytRam commented 1 year ago

Hi @pckyn,

Could you please provide detailed steps of how you added multiple PVA skills to the Composer bot?

  1. Add PVA bot skills in azure bot, may be please add two to three PVA skills to test the usecase
InfinytRam commented 1 year ago

Closing due to inactivity. @pckyn, please re-open with the details needed. Thanks.