search

microsoft / BotFramework-Emulator

A desktop application that allows users to locally test and debug chat bots built with the Bot Framework SDK.
https://aka.ms/botemulator
MIT License
1.81k stars 752 forks source link

QnAMaker trace can't be shown in Bot Emulator. #2257

Open axelsrz opened 3 years ago

axelsrz commented 3 years ago

Version

4.11.0

Describe the bug

Using Python bot with QnAMaker service, the QnAMaker trace can't be shown in Bot Emulator. However using DotNetCore bot with QnAMaker service, the QnAMaker trace can be shown in Bot Emulator.

To Reproduce

When using the Python QnAMaker sample bot, the QnAMaker trace encountered the error in Bot Emulator. image

Expected behavior

The QnAMaker trace can be shown: image

Zerryth commented 3 years ago

@axelsrz I did a little bit of digging by running sample 11.qnamaker for C#, JS, and Python bots. All of which resulted in the trace info not being shown in Emulator

image

Additionally, it appears that QnAMaker does seem to actually have the correct code to send a trace activity when QnAMaker.get_answers(...) method is called: https://github.com/microsoft/botbuilder-python/blob/13a9b096424e355bb69dca730c75965f8064eeb9/libraries/botbuilder-ai/botbuilder/ai/qna/utils/generate_answer_utils.py#L204

Maybe there's broken code within the SDK with the TurnContext.send_activity code or I believe the issue is actually within the Emulator instead. I'll keep hunting into LUIS to compare, since we know we can get LUIS traces successfully and in theory both QnA and LUIS should be emitting traces the same way

Things to note

Looking at the error message you see when you click on QnA Maker Trace in Emulator:

"Unable to find a QnA Maker service with Knowledge Base ID 7b13xxxx-xxxx-xxxx-xxxx-xxxxxxxx511e. Please add a QnA Maker service to your bot."

Seems like a strange error that may be a remnant artifact of our once-used .bot files? Because are we not adding QnAMaker to our bot if it's in our config.py? If I had to guess, never having looked at Emulator code before, so take what I say with a grain of salt, if we're doing "everything right" on the SDK end, the issue might be in the emulator, perhaps relying on some .bot-file-based logic? Iunno.

Zerryth commented 3 years ago

Additionally, forgot to add, trying to repro the user's steps, I tried going to Emulator > Services > + > Add QnA Maker as they have done, it prompts you to sign into Azure account, then it breaks

image

Zerryth commented 3 years ago

Ok. Confirmed that LUIS is sending traces the same way as is done in QnA Maker packages: https://github.com/microsoft/botbuilder-python/blob/c85bafaacda5aa44d61fafa39927db74047fc060/libraries/botbuilder-ai/botbuilder/ai/luis/luis_recognizer_v3.py#L286

The SDK creates a trace activity, then uses TurnContext.send_activity to send the trace (in both LUIS and QnA). Despite this, LUIS traces are successfully viewable in Emulator, whereas QnA traces are not. Transferring this to Emulator repo