Open cwhitten opened 1 year ago
Hi, Thank you so much for your information @cwhitten. How about if any plan for Migrate from Language Understanding (LUIS) to conversational language understanding (CLU), that can work for Bot Composer?
btw, one of your C# specific links points to JS... although the correct link is also listed at top
@cwhitten will there be support for CLU and its multilingual capabilities?
CLU guidance will be coming in the months ahead. We are actively working with the Cognitive Services Language Team on the best guidance as it relates to Composer / SDK bot building. We will be providing communication once that guidance is finalized.
CLU guidance will be coming in the months ahead. We are actively working with the Cognitive Services Language Team on the best guidance as it relates to Composer / SDK bot building. We will be providing communication once that guidance is finalized.
Thanks @cwhitten and @Dewain27 for the update about the custom CQA recognizer and potential future updates on CLU.
We are currently developing a new dialog architecture of our existing "flow-bot", written in Composer (*) and want to use CLU & Question Answering. We are in a multilingual setup with 4 languages and would love to have something similar to the existing LUIS/QnA solution with a recognizer set and some build-in support for CLU.
@Dewain27 Do you already know if there will be some built-in solutions or "extensions" like a custom recognizer?
In the meantime, would you recommend:
Building our own custom recognizer? Can be implemented like custom actions. Currently, there is no good documentation, but some examples can be found here: iciclecreek.bot/source/Libraries at master · tomlm/iciclecreek.bot · GitHub and there is the mentioned implementation for the Bot Framework SDK.
Building a custom middleware, which will capture the out-going request to LUIS and calls the CLU Rest API instead and stores the results accordingly. (So BFC thinks its based on LUIS)
Something else (or waiting for the provided solution)
(*) PVA is currently no valid option for us, as we do not have the procurement basis and need customization like connecting via mTLS to an on-premise network.
@primle also check out the usage of custom recognizer in these docs: https://learn.microsoft.com/en-us/composer/concept-language-understanding?source=recommendations&tabs=v2x#to-add-language-understanding-to-a-dialog
Hi what is the update on this one @cwhitten your original post did say "coming weeks" back on the 12th of Oct, it's the end of November. What is the deal.
Hi! Is there any update on this issue? We really need to make a decision about what tools we're gonna use moving fodward.
I have Custom QA working fine via API calls from my Composer bot project, and processing of multi-prompts. Just Regular Expression recognizer and Custom Question Answering. HOWEVER, can NOT PUBLISH to an Azure Bot, as it continues to ask for LUIS language understanding /authoring and QnAmaker resources. As these are not used in bot (they are just default/dummy values in Bot Composer appsettings.json which works fine in Composer webchat or emulator.
How do we keep Publish from requiring actual LUIS and QNA resources? They are not used nor required.
The bot was buily from EMPTY BOT Template created from Bot Composer.
"luis": {
"authoringKey": "",
"authoringEndpoint": "",
"endpointKey": "",
"endpoint": "",
"region": "West Europe"
},
"qna": {
"subscriptionKey": "<subscription key>",
"endpoint": "<endpoint>"
},
"MicrosoftAppId": "<app id from Bot Framework registration>",
"MicrosoftAppPassword": "<app password from Bot Framework registration>"
Appears to make no difference whether I include or exclude the LUIS / QNA settings in publish profile... Publish tries to validate. So want to know how I disable them, as they are not used.
Hi @cwhitten , We are waiting from many weeks, it is so important for our plans using Microsoft Bot Composer with Enterprise level support. Can you support us with the new update?
Thanks.
@fausttiger, i feel your pain, I've been down this exact road ☹. To fix it, you need to remove all references to QnA knowledgebase from your composer settings/project. These include QnA dialogs (and perhaps triggers, i only had to do the dialogs) in places such as the chitchat dialog at the parent level and any other dialogs referencing QnA don't forget the chitchat dialog in Bot responses section as well. You may need to go into the directory containing the bot project and do the same. Note until you remove the content of the chitchat dialog under your bot responses the issue will not go away. I found this part a bit tricky as the problem persisted even after i'd deleted the chit chat responses within the Bot response section on the canvas, in the end i had to open the file within the bot project folder, where I selected and deleted all the contents. This did the trick.
You may have already done this but check that after doing all of the above, ensure that QnA resource is deselected from the resources you want to provision.
Hi everyone,
Thank you for your patience as we're worked through this. We're very close to enabling a CQA integration through a new Bot Framework component package. A PR is in review here: https://github.com/microsoft/botframework-components/pull/1405, afterwhich we'll publish it to a public nuget feed.
@Intelliteckai
You mentioned QnAmaker refs, but did you remove your LUIS references also or are still using LUIS (not really needed with CustomQA and Regex)
Not sure which references to QnA or LUIS... Default Recognizer for bot is Regular Expression Recognizer for all dialogs is Regular Expression.
no dialogs reference QnA... (I have no ChitChat dialog as this was based on emptyBot template). I think LUIS and QnA references are just generated from the code built into Composer that has configuration for them even if not used.
There are (blank empty) files like
CQA_Bot.en-us.qna or CQA_Bot.en-us.lu
in folders such as
knowledge-base or language-understanding,
but those are all empty (but maybe still referenced)
removing unused settings from appsettings.json doesn't do it either.
I'l note the EmptyBot template created from Composer isn't even supposed to have QnA or LUIS components (but it still requires (or validates) them for Publish)
@cwhitten
this is still only for a full code solution, and not a composer based solution?
I have my composer based solution working, but can't deploy to azure due to dependencies wired into Composer it seems.
Hi everyone,
Thank you for your patience as we're worked through this. We're very close to enabling a CQA integration through a new Bot Framework component package. A PR is in review here: microsoft/botframework-components#1405, afterwhich we'll publish it to a public nuget feed.
I have Custom QA working fine via API calls from my Composer bot project, and processing of multi-prompts. Just Regular Expression recognizer and Custom Question Answering. HOWEVER, can NOT PUBLISH to an Azure Bot, as it continues to ask for LUIS language understanding /authoring and QnAmaker resources. As these are not used in bot (they are just default/dummy values in Bot Composer appsettings.json which works fine in Composer webchat or emulator.
How do we keep Publish from requiring actual LUIS and QNA resources? They are not used nor required.
The bot was buily from EMPTY BOT Template created from Bot Composer.
"luis": { "authoringKey": "", "authoringEndpoint": "", "endpointKey": "", "endpoint": "", "region": "West Europe" }, "qna": { "subscriptionKey": "<subscription key>", "endpoint": "<endpoint>" }, "MicrosoftAppId": "<app id from Bot Framework registration>", "MicrosoftAppPassword": "<app password from Bot Framework registration>"
Appears to make no difference whether I include or exclude the LUIS / QNA settings in publish profile... Publish tries to validate. So want to know how I disable them, as they are not used.
@fausttiger would love to see how you got this set up & working. Could you share your set-up / screenshots?
@fausttiger the approach for QnA is the same for LUIS. I decided to continue using LUIS instead of CLU because it is more complicated to setup than QnA if you consider that you have to create http actions for each intent in trigger in LUIS. Also LUIS has not been taken offline like the QnA service. You can see from the attached image that for the unknown intent where i called the http action, i defined a multi switch options for the two possible utterances including the default, (Help and None). To set it up for language, you will need to add all your intents in the switch and then follow it up with the actions for the bot to take. If you consider the way the LUIS works, the users says something (utterance), it gets sent to the LUIS to determine the intent from the keyphrase. The LUIS returns the probable intent (top intent) confidence thresholds score to the bot, the bot then looks up what to do next as defined in the dialog. it's the same here.
I have Custom QA working fine via API calls from my Composer bot project, and processing of multi-prompts. Just Regular Expression recognizer and Custom Question Answering. HOWEVER, can NOT PUBLISH to an Azure Bot, as it continues to ask for LUIS language understanding /authoring and QnAmaker resources. As these are not used in bot (they are just default/dummy values in Bot Composer appsettings.json which works fine in Composer webchat or emulator. How do we keep Publish from requiring actual LUIS and QNA resources? They are not used nor required. The bot was buily from EMPTY BOT Template created from Bot Composer.
"luis": { "authoringKey": "", "authoringEndpoint": "", "endpointKey": "", "endpoint": "", "region": "West Europe" }, "qna": { "subscriptionKey": "<subscription key>", "endpoint": "<endpoint>" }, "MicrosoftAppId": "<app id from Bot Framework registration>", "MicrosoftAppPassword": "<app password from Bot Framework registration>"
Appears to make no difference whether I include or exclude the LUIS / QNA settings in publish profile... Publish tries to validate. So want to know how I disable them, as they are not used.
@fausttiger would love to see how you got this set up & working. Could you share your set-up / screenshots?
@nickfrancis are there any .qna files in your project? You should be able to delete the knowledge-base
directory in your project if so. If you have no publishing profile that provisioning QnAMaker I don't see why it would attempt to publish any QnA resources. Please try this
@cwhitten
I'll see what I can post.
Trying to get Publish from Bot to work (talking with MS shortly) as can't yet publish from Composer to Azure via ImportResources. (MS person mentions Composer bug, Composer assuming types of bots/resourcs to be deployed, but don't have details).
Basically -I'm using API POST to call custom question answering. -If MultiPrompt count nonzero.. then Additional process to retrieve Prompt name(s) and QNAid for each, and present to user (HeroCard),
QUESTION Does Regular Expression evaluation use LUIS? (so if all recognizers ar Regex is LUIS still used?). As I don't specify any LUIS in my local Bot, I wouldn't think so.
@fausttiger would love to see how you got this set up & working. Could you share your set-up / screenshots?
If you consider the way the LUIS works, the users says something (utterance), it gets sent to the LUIS to determine the intent from the keyphrase. The LUIS returns the probable intent (top intent) confidence thresholds score to the bot, the bot then looks up what to do next as defined in the dialog. it's the same her
@Intelliteckai ,
if only Regex recognizer used for recognizer, is LUIS still used?
Unknown intent can't use LUIS as I have none defined. I never use Default REcognizer only regex (including Cancel/Help/etc. dialogs)
But yes, I might just add an unused LUIS resource if needed to avoid Publish validation issues (bot doesn't require it)
are there any .qna files in your project? You should be able to delete the
knowledge-base
directory in your project if so. If you have no publishing profile that provisioning QnAMaker I don't see why it would attempt to publish any QnA resources. Please try this
@cwhitten
does same apply for .LU files? There are a number in my project, but they are all empty files.
i.e. Bot_CQA\language-understanding\en-us\CQA_Bot.en-us.lu Bot_CQA\knowledge-base\en-us\CQA_Bot.en-us.qna etc.
OK, with just Custom QA and Regex recognizers we got the bot deployed from Composer into Azure Bot.
Our current bot project was already working find in Bot Composer web chat or emulator... we just couldn't Import Resources (created all in Azure) to publish bot code to Azure Bot.
We basically just had composer create basic bot requirements + LUIS; Had language create language components, and manually add some reporting related elements in Azure
This is how we provisioned/published (I'll have to get some details how we have Customer QA working in our bot composer project later)...
1) From Azure Create Language project - Custom Question Answering which creates Search service Storage account
2) From Composer (my working project) new publish profile / create resources (just bot, required and LUIS, no QnA) Required Defaults -Azure Bot -App Service plan -App Service Plus Options: -Luis Authoring -Luis Prediction
continue which provisions resources in Azure
3) From Bot Composer, publish code bot to Azure Works in Azure.
4) In Composer bot appsettings.json
5) Manually in Azure create -Application Insights -Log Analytics workspace add keys/etc. as needed to connect to Language/other resources
Still need to determine best setup/queries to get log analytics/application insights data for telemetry /reporting in Power BI as different compared with QnAMaker.
@cwhitten
Any update regarding this issue?
I am using Bot Composer, I tried to use the new package, but I have an issue
dialog error: Type Microsoft.Bot.Components.Recognizers.CustomQuestionAnsweringRecognizer not registered in factory.
@fausttiger would love to see how you got this set up & working. Could you share your set-up / screenshots?
@cwhitten
Basically this in Composer below is the main query and prompts processing... (I already posted how I created Azure resources with unused LUIS) This bot was built on top of Composer generated Empty Bot.
CustomQA Bot in Composer.
This includes a ChangeTopic dialog to get a CurrentTopic value from user prompt selection.
Only regex recognizer and CustomQA Language query from HTML Post.
I filter Custom QA via Metadata filter to currentTopic, Help, ChitChat with QA pairs that have metadata tags TOPIC:xxx for each.
Topic:Help being questions that are not topic specific ("that's not what I was asking", etc.)
unknown intent
HTTP POST METHOD = ${settings.customQuestionAnswer.LanguageEndpointPredictionURL}
where metadata filtered BODY = { "top": 1, "question": "${turn.activity.text}", "includeUnstructuredSources": true, "confidenceScoreThreshold": "${user.scoreThreshold}", "rankerType": "QuestionOnly", "answerSpanRequest": { "enable": true, "topAnswersWithSpan": 1, "confidenceScoreThreshold": "${user.scoreThresholdPrompt}" }, "filters": { "metadataFilter": { "logicalOperation": "OR", "metadata": [ { "key": "topic", "value": "${user.currentTopic}" }, { "key": "topic", "value": "chitchat" }, { "key": "topic", "value": "help" } ] } } }
. . . Check if good short answer and optionally display
Display top answer
Check if answer had metadata topic:help and query general help answer (which also calls Topic specific help question/answer)
. . . Check for presence of follow-on prompts and call dialog to process
. . .
= = = = = = = = =
Multi follow-on Prompts
including… convert 3 x N prompts array to 1 x N name and 1xN QnAid vectors for user selection (probably a tighter way to do this, but it works)
.
.
.
Get desired prompt from user, and query that specific QnA ID
BODY = { "qnaId": "${dialog.prompt.qnaId}", "question": "${dialog.prompt.displayText}", "includeUnstructuredSources": true, "answerSpanRequest": { "enable": true, "topAnswersWithSpan": 1, "confidenceScoreThreshold": "${user.scoreThresholdPromptShort}" } }
. . . Display answer and query if additional prompts exist in answer
Hi everyone.
Again, thank you very much for your patience. Yesterday we published a C#-based Bot Framework package that Composer users can install into their C#-based bot project created from Composer. Note: Node.js bots cannot install or use this package.
Please see the README.md for instructions on how to configure and use this custom recognizer, and if you have issues when following these instructions, open a github issue here, where the team will help troubleshoot.
Note that package enables CQA runtime inferencing for intent recognition, and any knowledge-base authoring will need to be done in the Language Studio. Additionally, using this CQA recognizer is not compatible with any existing knowledge-bases that are authored directly in Composer. You will still create and use the QnA Intent Recognized trigger on the dialog which you've enabled the custom CQA recognizer.
Thanks @cwhitten , Will try and let you know in case of any bugs
Hi everyone.
Again, thank you very much for your patience. Yesterday we published a C#-based Bot Framework package that Composer users can install into their C#-based bot project created from Composer. Note: Node.js bots cannot install or use this package.
![]()
Please see the README.md for instructions on how to configure and use this custom recognizer, and if you have issues when following these instructions, open a github issue here, where the team will help troubleshoot.
Note that package enables CQA runtime inferencing for intent recognition, and any knowledge-base authoring will need to be done in the Language Studio. Additionally, using this CQA recognizer is not compatible with any existing knowledge-bases that are authored directly in Composer. You will still create and use the QnA Intent Recognized trigger on the dialog which you've enabled the custom CQA recognizer.
![]()
Thanks @cwhitten I have done some tests and it's working as you described.
Has anyone been updating their analytics to digest the Application Insights (workspace not old classic) and Log Analytics Workspace telemetry to work with the new Custom Question Answering or CLU?
we previously modified the Virtual Assistant Analytics template https://microsoft.github.io/botframework-solutions/solution-accelerators/tutorials/view-analytics/1-intro/ with new Workspace App Insights, however the table names have changed, so out of the box things break, but working to remedy that. Also separate api.loganalytics.io queries to Language Resource" and "Azure Bot resource" are required now it seems as data is still separated and needs to be joined.
Hi @cwhitten and @spmoreno , We have tested and it works well, so far we have not received any error. But I have 2 more questions:
Thanks.
Hi, Is there any update how to use the new CLU together with CQA?
Hi @cwhitten and @spmoreno , We have tested and it works well, so far we have not received any error. But I have 2 more questions:
- When can we remove the settings for QnA on the Configuration settings and the Publish Profile?
- How is about the CLU for Bot Framework Composer?
Thanks.
Hi @cwhitten , could you help check these questions mentioned by @nghivo ?
This was really easy to implement - thank you @cwhitten
In terms of moving LUIS over - is it just a case of importing & using standard "Intent recognized" triggers within Bot Framework Composer?
@fausttiger am also needing to look at analytics. Was using the same template you linked but haven't done any modification to it.
@nickfrancis
Yes I used the Virtual Assistant power bi template with my previous QnAMaker bot... connecting to the Application Insights... (it's tagged as not supported anymore)
but Table names and Field names have changed... so much of that is broken..
i.e. CustomEvents --> AppEvents, etc...
(See
https://learn.microsoft.com/en-us/azure/azure-monitor/app/convert-classic-resource#workspace-based-resource-changes
for legacy names, vs log analytics vs application insights )
.
.
.
Against the LANGUAGE resource (not Azure Bot) These Kusto queries work (save as Power BI query after running in Azure), but are limited in scope of information.
https://learn.microsoft.com/en-us/azure/cognitive-services/language-service/question-answering/how-to/analytics
kusto will convert to power query like
= let Source = Json.Document(Web.Contents("https://api.loganalytics.io/v1/workspaces/xxxxxx-yyyy-zzzzz-aaaa-bbbbbbb/query", [Query=[#"query"=" AzureDiagnostics | where ResourceProvider == ""MICROSOFT.COGNITIVESERVICES"" | where OperationName==""CustomQuestionAnswering QueryKnowledgebases"" | extend answer_ = tostring(parse_json(propertiess).answer) | extend question = tostring(parse_json(propertiess).question) | extend score = tostring(parse_json(propertiess).score) | extend kbId = tostring(parse_json(propertiess).kbId) | project question, answer, score, kbId_ ",#"x-ms-app"="OmsAnalyticsPBI",#"timespan"="P3D",#"prefer"="ai.response-thinning=true"],Timeout=#duration(0,0,4,0)])), TypeMap = #table( { "AnalyticsTypes", "Type" }, { { "string", Text.Type }, { "int", Int32.Type }, { "long", Int64.Type }, { "real", Double.Type }, { "timespan", Duration.Type }, { "datetime", DateTimeZone.Type }, { "bool", Logical.Type }, { "guid", Text.Type }, { "dynamic", Text.Type } }), DataTable = Source[tables]{0}, Columns = Table.FromRecords(DataTable[columns]), ColumnsWithType = Table.Join(Columns, {"type"}, TypeMap , {"AnalyticsTypes"}), Rows = Table.FromRows(DataTable[rows], Columns[name]), Table = Table.TransformColumnTypes(Rows, Table.ToList(ColumnsWithType, (c) => { c{0}, c{3}})) in Table . . . Not sure yet if I need to generate custom telemetry events from composer
to capture certain items... .... and which table/etc. this will end up in the new bot logs. And, will have to somehow join Language telemetry to bot/application insights telemetry. . . . BTW, be sure to set the telemetry settings to True (if no sensitive data in user exchanges) or not all data is output to logs.
I'm not there yet, and find little examples online tha reflect the new bot and custom question answering scenario.
nickfrancis commented
@fausttiger am also needing to look at analytics. Was using the same template you linked but haven't done any modification to it.
hi was doing some work on testing a CLU imported from LUIS.
I noticed that when adding a new Intent Recognized trigger there is no Trigger Phrases editor:
Is there a way of resolving this?
Hi everyone.
Again, thank you very much for your patience. Yesterday we published a C#-based Bot Framework package that Composer users can install into their C#-based bot project created from Composer. Note: Node.js bots cannot install or use this package.
![]()
Please see the README.md for instructions on how to configure and use this custom recognizer, and if you have issues when following these instructions, open a github issue here, where the team will help troubleshoot.
Note that package enables CQA runtime inferencing for intent recognition, and any knowledge-base authoring will need to be done in the Language Studio. Additionally, using this CQA recognizer is not compatible with any existing knowledge-bases that are authored directly in Composer. You will still create and use the QnA Intent Recognized trigger on the dialog which you've enabled the custom CQA recognizer.
![]()
Thanks @cwhitten & @spmoreno, its working well.... BUT Why its not working when I create it using "Add a Dialog"? When I call this dialog using "Begin a new dialog" only "BeginDialog" works, "QnA Intent recognized" doesn't works. There is no error. Also json is added and the package is installed.
Does it will only work with start dialog or is there any way we can create the same thing within another dialog and call it when needed??
hey is there any update to migrating LUIS to CLU . we are building an enterprise level chatbot . and it doesn't make sense to build it now in January where it will stop in April ( when it comes to creation) can we have a guideline on migrating LUIS to CLU please
We would also like any update on migrating LUIS to CLU, specifically for Composer built bots. Is it feasible to create a custom recogniser for CLU and is their any documentation on how to approach this?
@cwhitten same issue here as mentioned by itsmeprateek, the qna trigger is not triggered if the dialog is invoked from another dialog. Is there any way to make the solution work for a multi dialog bot?
@nickfrancis @fballan93 @Charlie2922 Hi!
In case you haven't seen it, CLU can be integrated with BFC as explained here.
Configuring expressions and intents is done directly in the Language Studio portal. In Composer, in addition to the package configuration, a trigger (Intent recognized type) must be generated for each intent, with the same name as it is in the portal.
However, I am now working on solving other needs, such as capturing the response score in a variable or presenting the topIntent as a response.
Latest (stable) release: Feb 3, 2022...
12/14/22 update: See this link for more information on using a custom CQA recognizer: https://github.com/microsoft/BotFramework-Composer/issues/9400#issuecomment-1351756216
Hello everyone,
With the recent announcements around the retirement of QnA Maker and GA release of CQA (Custom Question Answering), we wanted to provide some guidance to our community. We have a sample for using CQA with the BF (Bot Framework) SDK today and the link is below and for Composer we are looking to provide a package that will allow for CQA as a custom recognizer in the coming weeks. To provide some relief, we have extended the ability to create new QnA resources temporarily for those subscriptions where we identified active use of Composer for QnA authoring between 9/1/2022 and 10/1/2022. The extension will be provided to these customers until 1/1/2023. Premier Support Customers that feel that they need more time to provision QnA Maker resources that were not already approved will be required to open a support request to ask for an exception that can be evaluated by our PG for approval. We will be communicating guidance on moving away from QnA Maker for BF Composer as a resource in the coming weeks as there are great advantages to our new product offering.
Here are some documentation links that are available currently and we will post again when we have more to share:
SDK CQA Sample
BotBuilder-Samples/samples/csharp_dotnetcore/12.customQABot at main · microsoft/BotBuilder-Samples (github.com)
BotBuilder-Samples/samples/csharp_dotnetcore/48.customQABot-all-features at main · microsoft/BotBuilder-Samples (github.com)
JavaScript
https://github.com/microsoft/BotBuilder-Samples/tree/main/samples/javascript_nodejs/12.customQABot
[https://github.com/microsoft/BotBuilder-Samples/tree/main/samples/javascript_nodejs/48.customQABot-all-features]()
Csharp
BotBuilder-Samples/samples/csharp_dotnetcore/12.customQABot at main · microsoft/BotBuilder-Samples (github.com)
BotBuilder-Samples/samples/csharp_dotnetcore/48.customQABot-all-features at main · microsoft/BotBuilder-Samples (github.com)
CQA Documentation
[What is question answering? - Azure]()
How to migrate QnaMaker Cognitive Services - Microsoft Learn
Question Answering tutorial
We will continue to collect your feedback and share updates to this pinned github issue as we have them. Thank you