Direct Line App Service Extension does not work with Azure bot created with Virtual Assistant Template

[pravinambekar]

What project is affected?

VA template based Azure web app bot

What language is this in?

C# Dot Net Core

What happens?

Azure Bot created with Virtual assistant template running in azure environment working fine in Web Chat and Direct line (When embedded as IFrame) but when i enabled Direct Line App Service Extension (https://docs.microsoft.com/en-us/azure/bot-service/bot-service-channel-directline-extension?view=azure-bot-service-4.0) i am getting Web Socket failed Error :

WebSocket connection to 'wss://xxx.azurewebsites.net/.bot/v3/directline/conversations/connect?token=ew0KICAi...' failed:

What are the steps to reproduce this issue?

1) Created VA based Bot and deployed on Azure service (QnA based bot) 2) Enabled Direct line channel in Web App Bot 3) Enabled Enhanced authentication options and added trusted origin urls 4) Configured App Service Extension Key in App service 5) Enabled Web Socket option from configuration page in app service 6) Did required changes in Bot application source code to add WebSocket and Named pipe option as below

 app.UseWebSockets();
 app.UseNamedPipes(System.Environment.GetEnvironmentVariable("APPSETTING_WEBSITE_SITE_NAME") + ".directline");

7) Created api method to generate token using

HttpRequestMessage request = new HttpRequestMessage(
                HttpMethod.Post,
                $"https://directline.botframework.com/v3/directline/tokens/generate");

8) Created Client html page and requested for token generated from above snippet and rendered web chat using below snippet

 window.WebChat.renderWebChat(
          {
            directLine: await window.WebChat.createDirectLineAppServiceExtension({
              domain: 'https://xxx.azurewebsites.net/.bot/v3/directline',
              token
            })
          },
          document.getElementById('webchat')
        );

What were you expecting to happen?

The client page should render web chat and start communication like it works in normal IFrame or Test in Web Chat Option

Can you share any logs, error output, etc.?

webchat.js:2 WebSocket connection to 'wss://xxx.net/.bot/v3/directline/conversations/connect?token=ew0KICAiYWxnIjogIlJTMjU2IiwNCiAgImtpZCI6ICJ2cG4y... failed:

Any screenshots or additional context?

Hello @lauren-mills , @Batta32 any help here would really appreciated , i found similar issue here : https://github.com/microsoft/botframework-solutions/issues/3780#issuecomment-822754620 but even all the code and configuration is same, i am still getting websocket fail error.

[YutongTie-MSFT]

@dmvtech Hi Dana, I am tracking this issue from Microsoft Q&A platform. Could you please help take a look? Thanks.

[dmvtech]

I will take a look.

[pravinambekar]

I will take a look. @dmvtech Hello Dana, Did you got time to look into this, i really failed to understand root cause for the issue, at one side Test In Web chat works fine, i have a feeling that there is some issue with the way client side request is communicating with bot over direct line and the definition for createDirectLineAppServiceExtension function defined in botframework-webchat js as the constant error : WebSocket connection to XXX failed. is there and there is literally nothing i could fix about this.

Moreover calls to https://directline.botframework.com/v3/directline/conversations working fine in Postman, it means secrete and bot app is correctly configured. And even the client side code is simple and as per Bot Framework documentation. index.txt

another question : The token i generated have issuer and Audience as : "iss": "https://directline.botframework.com/", "aud": "https://directline.botframework.com/"

Where as documentation from: https://github.com/microsoft/BotFramework-WebChat/tree/main/samples/01.getting-started/i.protocol-direct-line-app-service-extension says it should be : The token is a JSON Web Token and the iss and aud fields are both https://directlineextension.botframework.com/ https://directlinextension.botframework.com

Does that make any difference,, if yes, how can we get directlineextension as iss and aud ?

It would be really helpful if get cause for the issue. Thank you again.

[pravinambekar]

I will take a look. @dmvtech Hello Dana, Did you got time to look into this, i really failed to understand root cause for the issue, at one side Test In Web chat works fine, i have a feeling that there is some issue with the way client side request is communicating with bot over direct line and the definition for createDirectLineAppServiceExtension function defined in botframework-webchat js as the constant error : WebSocket connection to XXX failed. is there and there is literally nothing i could fix about this.

Moreover calls to https://directline.botframework.com/v3/directline/conversations working fine in Postman, it means secrete and bot app is correctly configured. And even the client side code is simple and as per Bot Framework documentation. index.txt

another question : The token i generated have issuer and Audience as : "iss": "https://directline.botframework.com/", "aud": "https://directline.botframework.com/"

Where as documentation from: https://github.com/microsoft/BotFramework-WebChat/tree/main/samples/01.getting-started/i.protocol-direct-line-app-service-extension says it should be : The token is a JSON Web Token and the iss and aud fields are both https://directlineextension.botframework.com/ https://directlinextension.botframework.com

Does that make any difference,, if yes, how can we get directlineextension as iss and aud ?

It would be really helpful if get cause for the issue. Thank you again.

---------------Update-----------------------

I continued investigation and have tried to generate token from my own app service url (https://xxx.azurewebsites.net/.bot/v3/directline/tokens/generate) in token controller but the call returns null or empty token and when i check log in portal i am getting 404 error for the service call :

Could you please suggest why i am getting 404. ?

[SSanjeevi]

@pravinambekar - I have tried the same and it seems to be working - please refer my repo with complete sample, if you see any issues let me know : https://github.com/SSanjeevi/VirtualAssistantDirectlineExtn

You can check the app service log stream window when accessing the webchat, you will see the logs if you see any issue - by following my repo code for serilog logging.

[pravinambekar]

@pravinambekar - I have tried the same and it seems to be working - please refer my repo with complete sample, if you see any issues let me know : https://github.com/SSanjeevi/VirtualAssistantDirectlineExtn

You can check the app service log stream window when accessing the webchat, you will see the logs if you see any issue - by following my repo code for serilog logging.

Hello @SSanjeevi ,

Thank you for sharing repo, while checking your source i am getting serilog error as below :

[ERR] Some services are not able to be constructed 
(Error while validating the service descriptor 'ServiceType: Microsoft.Bot.Builder.Integration.IAdapterIntegration 
Lifetime: Singleton ImplementationType: VirtualAssistantSample.Adapters.DefaultAdapter': 
Unable to resolve service for type 'Microsoft.Bot.Connector.Authentication.AuthenticationConfiguration' 
while attempting to activate 'VirtualAssistantSample.Adapters.DefaultAdapter'.) 
(Error while validating the service descriptor 'ServiceType: Microsoft.Bot.Builder.Integration.AspNet.Core.IBotFrameworkHttpAdapter 
Lifetime: Singleton ImplementationType: VirtualAssistantSample.Adapters.DefaultAdapter': 
Unable to resolve service for type 'Microsoft.Bot.Connector.Authentication.AuthenticationConfiguration' 
while attempting to activate 'VirtualAssistantSample.Adapters.DefaultAdapter'.) 
(Error while validating the service descriptor 'ServiceType: Microsoft.Bot.Builder.ChannelServiceHandler 
Lifetime: Singleton ImplementationType: VirtualAssistantSample.TokenExchange.TokenExchangeSkillHandler': 
Unable to resolve service for type 'Microsoft.Bot.Solutions.Skills.SkillsConfiguration' 
while attempting to activate 'VirtualAssistantSample.TokenExchange.TokenExchangeSkillHandler'.)   
at Microsoft.Extensions.DependencyInjection.ServiceProvider..ctor(IEnumerable`1 serviceDescriptors, ServiceProviderOptions options)

Is this related to version issue ? i have seen similar error you reported on SO, but the thread seems not have concrete soultion, may be you could help here. (while i will keep on investigating)

additionally I am bit concerned with the way client code is implemented, here we are directly exposing Direct Line Secret. which is suppose to be generate in separate api controller

Do you have any suggestion over the error i am getting during DI, may be token part we can take care in controller

Regards, Pravin

[SSanjeevi]

Is that error coming in log stream console? - may be you missed this second line I think not sure:

services.AddSingleton<IBotFrameworkHttpAdapter, DefaultAdapter>(); services.AddSingleton<BotAdapter>(sp => sp.GetService<BotFrameworkHttpAdapter>());

for webchat - I done basic implementation - token you can do after this works.

[pravinambekar]

Is that error coming in log stream console? - may be you missed this second line I think not sure:

services.AddSingleton<IBotFrameworkHttpAdapter, DefaultAdapter>(); services.AddSingleton<BotAdapter>(sp => sp.GetService<BotFrameworkHttpAdapter>());

for webchat - I done basic implementation - token you can do after this works.

Hello @SSanjeevi , I was able to run bot locally but i guess there is some issue with running bot on local IIS (getting timeout when run on local iis but was able to run on console) In spite of that i deployed bot and related services on azure but getting 502 error upon access app service.

ModuleName  AspNetCoreModuleV2
Notification    EXECUTE_REQUEST_HANDLER
HttpStatus  502
HttpReason  Bad Gateway
HttpSubStatus   5
ErrorCode   The app didn't start in the required time. (0x8027025a)

Do you have any idea what possibly could go wrong ? I guess something is wrong with IIS configuration, or the way app is running on IIS

[SSanjeevi]

Is your /xxx.azurewebsites.net/.bot URL works? please check connecting from webchat client.

[pravinambekar]

Is your /xxx.azurewebsites.net/.bot URL works? please check connecting from webchat client.

Hi @SSanjeevi , The https:/xxx.azurewebsites.net/.bot is now returning json response : { v: "1.0.0.0.55fa54091a1e3b168628d4000ee2b06a88bbc2ac", k: true, ib: true, ob: true, initialized: true }

but while testing it from web chat client i am getting Failed to fetch error

chat.html:49 TypeError: Failed to fetch
    at chat.html:34:27
    at chat.html:49:9

[SSanjeevi]

check browser console - net work tab - you may have to check that and restart app service and enable socket.

[dmvtech]

Hi @pravinambekar

Looks like you've made some progress. Some additional questions:

1. I'm assuming since this is a VA bot that it is using BotFrameworkAdapter and not the newer CloudAdapter, correct?
2. You have correctly set APPSETTING_WEBSITE_SITE_NAME in the configuration?
3. When setting your secret for the bearer token, you are using the actual Directline secret (even though it is posting to your ASE generate endpoint) and not the Directline ASE secret, correct?
4. Can you clarify on:

but while testing it from web chat client i am getting Failed to fetch error (chat.html:49 TypeError: Failed to fetch)

What is on line 49? Checking the console per @SSanjeevi is a great idea.

[pravinambekar]

Is your /xxx.azurewebsites.net/.bot URL works? please check connecting from webchat client.

Hi @pravinambekar

Looks like you've made some progress. Some additional questions:

1. I'm assuming since this is a VA bot that it is using BotFrameworkAdapter and not the newer CloudAdapter, correct?
2. You have correctly set APPSETTING_WEBSITE_SITE_NAME in the configuration?
3. When setting your secret for the bearer token, you are using the actual Directline secret (even though it is posting to your ASE generate endpoint) and not the Directline ASE secret, correct?
4. Can you clarify on:

but while testing it from web chat client i am getting Failed to fetch error (chat.html:49 TypeError: Failed to fetch)

What is on line 49? Checking the console per @SSanjeevi is a great idea.

Hello @dmvtech , Hello @SSanjeevi ,

1) Yes VA bot that it is using BotFrameworkAdapter and not Cloud Adapter 2) Yes APPSETTING_WEBSITE_SITE_NAME is correctly configured 3) Yes The secrete used for token generated in Direct Line Secrete and not ASE secrete

About the error in console , I dig down further and found that the timeout is happening upon calling token generation service : https://XXX.azurewebsites.net/api/directline/token, i am yet to find reason why timeout is happening but meanwhile i tried skipping token call (Just for Test purpose) and directly provided Direct Line secrete as token to renderWebChat() method and could see Web chat rendered successfully.

Few concerns still to address : 1) Adaptive card attachment does not work directly 2) Refresh token logic

Do you have any idea why request token gets timeout, is this related to app service hosted environment or plan ?

[SSanjeevi]

@pravinambekar - thanks for confirming that bot works when using secret in webchat. Not sure about issue when using the token. What I analyzed(understood) is that its using WebSocket connection to connect between client and server using ASE - not sure whether we can create token before that socket connection is established - @dmvtech and team will help you on that.

[pravinambekar]

Not sure about issue when using the token. What I analyzed(understood) is that its using WebSocket connection to connect between client and server using ASE - not sure whether we can create token before that socket connection is established - @dmvtech and team will help you on that.

Hello @SSanjeevi , @dmvtech

I think i found the culprit, the url we use for token generation has to be : https://XXX.azurewebsites.net/.bot/v3/directline/tokens/generate and not https://XXX.azurewebsites.net/api/directline/token/generate

It was my bad.

[SSanjeevi]

please close the issue - if its resolved 🥇 happy to help.

[pravinambekar]

Thanks @SSanjeevi & @dmvtech for your help.