HueBot sample: Skype bot cannot connect to the service using skype

[RomanMolchanov2017]

I followed instructions for registering a bot (https://docs.microsoft.com/en-us/bot-framework/portal-register-bot) and deploying a real-time audio-video bot (https://docs.microsoft.com/en-us/bot-framework/dotnet/bot-builder-dotnet-real-time-deploy-visual-studio) to run the HueBot sample (https://github.com/Microsoft/BotBuilder-RealTimeMediaCalling/tree/master/Samples/HueBot). But after my bot was added to my skype contacts I cannot make a video call to my bot. I always get a response like here: "You can't talk to this bot just yet. But we are working on it." And at the bot framework dashboard, I have the issues: "There was an error completing this request due to an internal channel failure." I connected the Visual Studio 2017 debugger to the published job worker and saw that the controller didn't receive any requests. Bot configuration: Messaging endpoint: https://dataartskypebots.cloudapp.net App ID: bf0f99b0-1836-4548-b1c6-6540ee25d191 Enable messaging: Yes; Enable autoplay of media cards: Yes; Enable calling: Yes; Real Time Media: Yes; Video (Audio alway enabled): Yes; Screen sharing: Yes; Webhook (for calling): https://dataartskypebots.cloudapp.net/api/calling/call Enable adding to group: Yes;

Could you help me what is incorrect in the bot configuration or maybe you have any other recommendations?

[MalarGit]

Huebot does not have a controller for messages. I will update the sample to include this controller soon. Meanwhile could you add one? You can use the "Test" tab in the bot registration's right corner to send a message and verify that the bot received the message. This will valid the authentication works. Could you also check if the instance is up and running in Azure?

[MalarGit]

You might want to upgrade to BotBuilder nuget v3.8.1 per https://github.com/Microsoft/BotBuilder/issues/2770

[RomanMolchanov2017]

As you suggested me, I upgraded the BotBuilder nuget to v3.8.1 and added a controller for messages. I got it "as is" from the Bot template for Visual Studio (see https://docs.microsoft.com/en-us/bot-framework/resources-tools-downloads). This is like a simple echo service.

I can communicate with my bot using Bot Framework Channel Emulator and debug this process using Visual Studio 2017. The emulator uses this url: https://dataartskypebots.cloudapp.net/api/messages.

I also can send a request for connection using the url for CallController: https://dataartskypebots.cloudapp.net/api/calling/call. The emulator doesn't support calls but at least the request is received by the CallController.OnIncomingCall method and I can see this using debugger.

But using "Test" tab I still have the error message: "There was an error sending this message to your bot." When I use Skype - my bot doesn't receive messages and calls at all. Any additional ideas?

[MalarGit]

I updated Huebot with the MessagesController and updated the nuget packages as well. Could you try with the latest bits. Also please verify the following

1. The following lines in app.config of worker role have been replaced with appropriate values <add key="BotId" value="$BotHandle$" /> <add key="MicrosoftAppId" value="$MicrosoftAppId$" /> <add key="MicrosoftAppPassword" value="$BotSecret$" />

2. the instance is up and running in Azure

3. the "test" tab in the registration portal should not throw an error message now.

[gfclaveria]

Were you able to get this sample working? It's quite painful to deploy.

[RomanMolchanov2017]

First of all, thank you for your assistance.

We have troubles to run your project "as is" because you uses Microsoft Azure Tools 2.8 that isn't supported by VS 2017. And when we open your solution we have to convert the project to Microsoft Azure Tools 2.9. After that, we have to make several manual changes because of the solution loses some dependencies. But we now in progress of running your solution.

Will keep you updated.

[gfclaveria]

Yes, that's true. I had that issue as well and I couldn't even downgrade the Azure tool version on the VS 2015 I'm using.

I was able to deploy the sample to the Azure Cloud by commenting those config setting related to Certificate. However, doing that causes the WorkerRole instance not functioning. It's always recycling or on busy/restarting, but that I think is the only way to get it deployed (if you don't have a certificate yet.) I manage to have the WorkerRole instance in Running status by un-commenting and putting the correct values on the certificate and serviceDnsName. Yey! I uses the certificate thumbprint that was created after the deployment of the sample app. I don't know if I'm on the right path of doing that and using that certificate coz it seems that the service is happily running.

Having that fixed and corrected and running, the issue that still exist, skype cannot make calls to the bot. "You can't talk to this bot just yet. But we are working on it." :-(

Hope you guys has information on how to resolve this or what i'm doing wrong.

[MalarGit]

@RomanMolchanov2017 We are working on converting to Azure SDK 2.9 and .NET 4.6.2. Just didn't have enough time before BUILD. You had mentioned that "we have to make several manual changes because of the solution loses some dependencies". Could you please elaborate on that?

@gfclaveria Certificate is a must. Without the certificate it will not be able to talk to any of our services. Did you change app.config in workerrole? In the Bot registration portal, could you use the 'Test' button to send a test message and see if that works.

[RomanMolchanov2017]

When VS 2017 converts your HueBot solution to the format that can be supported by Microsoft Azure Tools 2.9 it loses references to several libraries (Microsoft.WindowsAzure.Diagnostics.dll, Microsoft.WindowsAzure.ServiceRuntime.dll). But this fix isn't enough. We still couldn't publish your solution to Azure because during the publish process we have the error: Unhandled Exception: It is likely that certificate 'CN=huebot2017.cloudapp.net' may not have a private key that is capable of key exchange or the process may not have access rights for the private key. Please see inner exception for detail. Inner Exception: Invalid provider type specified. We created the certificate by following this document: https://docs.microsoft.com/en-us/azure/cloud-services/cloud-services-certs-create Now we're still looking for a solution.

By the way, before we opened this issue, we loaded HueBot solution and just fully recreated it from scratch, using NET Framework 4.6.2, Microsoft Azure Tools 2.9 and latest versions of all NuGet packages except System.IdentityModel.Tokens.Jwt. We also made corrections to InstallNETFX.cmd and deployed the project to Azure using NET Framework 4.6.2. And this version of service was deployed successful but we still have problems with Skype communication, including Bots' dashboard.

[MalarGit]

@RomanMolchanov2017 Are you creating a self-signed certificate? We do need a certificate from trusted CA. Once you have the certificate with private key (.pfx) certificate, you should upload them to Azure cloud service and update your service configuration with the key.

https://docs.microsoft.com/en-us/bot-framework/dotnet/bot-builder-dotnet-real-time-deploy-visual-studio covers the prerequisites and how to upload the certificate into an azure cloud service.

Before you request for a certificate, please look at the requirements in the "Certificate from a valid certificate authority section.

The bot needs to be configured with a valid certificate from a trusted Certificate Authority (CA). The Subject Name (SN) or the last entry of the Subject Alternative Name (SAN) of the certificate should be the name of the cloud service. Wild-card certificates are currently not supported. If a CNAME is used to point to the cloud service, the CNAME should be the SN or the last SAN entry of the certificate.

I will update the documentation for the sample with the above information. Thanks for your patience. Please let me know if run into any other issues.

[RomanMolchanov2017]

Yes, you are right. We use a self-signed certificate with the private key (.pfx) and we've already uploaded it to Azure using the instruction you pointed to.

Just to clarify.

We want to test this library for conforming our needs. Am I right that you would like us to create a certificate from trusted CA for this?

[MalarGit]

@RomanMolchanov2017 Yes. A trusted Cert is needed. WildCard certs are not supported and the SN or last entry in SAN should match cloud service name (xxx.cloudapp.net)..

[RomanMolchanov2017]

Thank you for clarifying.

[gerrosc]

@MalarGit Hi, yes I've modified the app.config and Test to send message. It didn't seem to work and getting this issue on the Web Chat health -- "There was an error sending this message to your bot." --

Any idea or recommendation/suggestion to were we could get that kind of trusted certificated? (or How perhaps)

I've tried to find websites(certificate authorities) to which I can acquire free dns/ssl certificate for let say "xxx.cloudapp.net", just testing purposes but couldn't get one. Most of the ones that I found have changed their policies regarding giving these kind of certificate.. they require that you have access the admin email address (e.g. admin@xxx.cloudapp.net or hostmaster@xxx.cloudapp.net) for verification code they will send before they create the certificate.

Thanks,

[MalarGit]

@RomanMolchanov2017 @gerrosc I was pointed to this website for getting certificates https://docs.microsoft.com/en-us/azure/app-service-web/web-sites-purchase-ssl-web-site. The pricing details are here.. https://azure.microsoft.com/en-us/pricing/details/app-service/. I have not tried this out yet. I will try in the next few days and update this thread. Let us know if you run into other issues. would be happy to help.

[gerrosc]

@MalarGit Thank you! :-)

[RomanMolchanov2017]

We've bought the certificate and now the skype can connect to our HueBot service. We've added the MessageController and message exchange works well. But the CallController still doesn't work. It can receive incoming requests from the skype but after that, we have issues like 'RealTimeMediaCallingConversation: System.ArgumentException: Empty Actions List not allowed' and 'RiffAudioContainer.ParseFormatChunk: nonstandard audio header encountered. Extra bytes length bytes missing'. Below attached logs' snapshot of calling session when the skype tries to call our HueBot service. We think it can be useful for you to localize the problem: HueBotSkypeCallLogs.txt

Any idea or recommendation/suggestion?

[MalarGit]

In the ServiceDnsName in the .cscfg have you used skypebot.dataart.com? we are trying to reach skypebot.dataart.com:20100 and the connection is refused. Could you try changing to dataartskypebots.cloudapp.net? Also what is the SN of the cert?

[RomanMolchanov2017]

We created certificate 'skypebot.dataart.com'. As the DNS name of our cloud service is 'http://dataartskypebots.cloudapp.net/' we also created CNAME DNS record skypebot.dataart.com -> dataartskypebots.cloudapp.net. When we tried to use 'skypebot.dataart.com' as ServiceDnsName in the .cscfg during deployment to Azure, we had the error: "Unhandled Exception: No such host is known". Therefore we changed the ServiceDnsName in the .cscfg back to 'dataartskypebots.cloudapp.net'. We also made following changes in the AzureConfiguration class: for creating CallControlCallbackUrl, NotificationCallbackUrl and MediaPlatformInstanceSettings.ServiceFqdn we used 'skypebot.dataart.com' instead of 'dataartskypebots.cloudapp.net'. At the bot's dashboard we defined following URLs:

Messaging endpoint: https://skypebot.dataart.com/api/messages

Webhook (For calling): https://skypebot.dataart.com/api/calling/call Enable calling, Real Time Media, Video (Audio always enabled), Screen sharing.

Now, we changed MediaPlatformInstanceSettings.ServiceFqdn back to 'dataartskypebots.cloudapp.net' as you requested, but we still have CallControlCallbackUrl and NotificationCallbackUrl defined with 'skypebot.dataart.com'.

Please, try to reach dataartskypebots.cloudapp.net:20100 now.

[MalarGit]

This was the error - The expected DNS identity of the remote endpoint was 'skypebot.dataart.com' but the remote endpoint provided DNS claim 'www.skypebot.dataart.com'. Was 'www.skypebot.dataart.com' the SubjectName of your cert? Could you try setting MediaPlatformInstanceSettings.ServiceFqdn as 'www.skypebot.dataart.com' to see if that fixes the problem?

As far as Unhandled Exception: No such host is known, the same config was used for resolving public ip and for callback urls. I will split the config and checkin a fix soon. Thanks.

[RomanMolchanov2017]

We changed MediaPlatformInstanceSettings.ServiceFqdn to 'www.skypebot.dataart.com' but that didn't resolve the problem.

Let's summarize the full stack configuration to define what is inconsistent:

SubjectName of our certificate: 'skypebot.dataart.com' (not 'www.skypebot.dataart.com'); DNS name of our cloud service is 'dataartskypebots.cloudapp.net'; We created CNAME DNS record skypebot.dataart.com -> dataartskypebots.cloudapp.net. Public IP address dataartskypebots cloud service at Azure dashboard: 52.166.115.5

nslookup skypebot.dataart.com Server: UnKnown Address: 2001:67c:438:11::15 Non-authoritative answer: Name: dataartskypebots.cloudapp.net Address: 52.166.115.5 Aliases: skypebot.dataart.com

At the bot's dashboard we have following configuration: Messaging endpoint: https://skypebot.dataart.com/api/messages Webhook (For calling): https://skypebot.dataart.com/api/calling/call Enable calling, Real Time Media, Video (Audio always enabled), Screen sharing

In the configuration of the service we have ServiceDnsName -> dataartskypebots.cloudapp.net DefaultCallControlEndpoint -> Internal=https://100.108.38.75:9440, PublicPort=443 InstanceMediaControlEndpoint -> Internal=https://100.108.38.75:8445, PublicPort=20100 CallControlCallbackUri -> https://skypebot.dataart.com:10100/api/calling/callback/ NotificationCallbackUrl -> https://skypebot.dataart.com:10100/api/calling/notification/ Call control listening Uri -> https://100.108.38.75:10100/ Call control listening Uri -> https://100.108.38.75:9440/ ServiceDnsName -> dataartskypebots.cloudapp.net ServiceDnsAlias -> skypebot.dataart.com Cloud Service Public IPAddress -> 52.166.115.5 Media Platform Instance Internal Port -> 8445 Media Platform Instance Public Port -> 20100 Media Platform Instance Public IPAddress -> 52.166.116.150 Service FQDN -> skypebot.dataart.com (we also tried these: www.skypebot.dataart.com, dataartskypebots.cloudapp.net) MicrosoftAppId -> bf0f99b0-1836-4548-b1c6-6540ee25d191 mpUri -> net.tcp://skypebot.dataart.com:20100/MediaProcessor

New log of one skype call that includes trace of service configuration is attached.

HueBotSkypeCallLogs24052017.txt

[MalarGit]

The settings look right.. Could you check if the last entry of the SAN in the certificate is www.skypebot.dataart.com? That takes precedence over the SN

[RomanMolchanov2017]

Yes, you're right. Our certificate has CN=skypebot.dataart.com and SAN =skypebot.dataart.com and www.skypebot.dataart.com.

[MalarGit]

Apart from modifying the MediaPlatformInstanceSettings.ServiceFqdn to 'www.skypebot.dataart.com', could you also change the CNAME to www.skypebot.dataart.com -> dataartskypebots.cloudapp.net? I will update the documentation to reflect that the last entry in SAN takes precedence. Thanks Roman

[RomanMolchanov2017]

Thank you for assistance Malar. The last configuration changes allowed the bot to receive calls from Skype.

When I make a call to HueBot it works as expected, without any troubles with echo and video.

But now we have several issues:

HueBot can't participate in Skype group video calls with multiple participants. We tested several cases:

1. When HueBot is added to the group video call we don't see any video in the HueBot area but have horrible echo during conversation.
2. When I create one-to-one Skype call to the HueBot and it answers my call, I add an additional participant to the conversation. Right after that, the HueBot leaves the conversation.

Nobody except me, from the group video call participants, can make the call to the HueBot, even in the one-to-one call mode. In this case, HueBot doesn't loopback user's video input, but it looks like it loopbacks audio as we have the same horrible echo and a black screen.

[ssulzer]

The HueBot was not intended to be used in group conversations, which is why it cannot see the video of any of the participants. Furthermore, bots do not support changing a 1:1 call with them into a group conversation: they will be removed from the conversation.

But more important to note is that at this time we cannot support calling bots in Skype group conversations, as the appropriate privacy controls are not yet ready in the clients. For now, only 1:1 calling bots may be developed.

Stephen Sulzer [MSFT]

[RomanMolchanov2017]

Thank you for the explanation, Stephen.

Do you have any preliminary estimation when you are going to implement following functionality:

Real-time media calling with Skype

... A real-time media bot may also participate in Skype group video calls with multiple participants. ... When joined to a multiparty Skype group video call, the real-time media bot can support receiving multiple main video streams simultaneously. This allows the bot to "see" more than one participant in the group video call. ... When joined to a Skype group video call consisting of multiple participants, the bot can identify which participants are currently speaking. Active speakers identify which participants are being heard in each received audio frame. Dominant speakers identify which participants are currently most active (or "dominant") in the group conversation, even though their voice may not be heard in every audio frame. The set of dominant speakers can change as different participants take turns speaking.

Thanks, Roman.

[RomanMolchanov2017]

It looks like if the issue is closed you don't answer questions. We reopen the issue just because we'd like to have the answer to our question.

Thanks, Roman.

[MalarGit]

Roman, The feature has been disabled for privacy reasons. It will be back up once we implement the appropriate privacy controls. We don't have an timeframe yet. We will remove that documentation and MeetingScreenShots sample till then.

[RomanMolchanov2017]

Thank you for the reply. It's also very important information for us.

[ghost]

@RomanMolchanov2017 Did you huebot works? I followed the instructions but when i call my bot it closes after 11 seconds.

I dont know if its because of my ServiceFQDN.

Also do I have to configure the ports in my solution?

[MalarGit]

@productivitySTK Could you provide more details on your bot? Do you have a valid certificate from a trusted CA. Is the messaging working fine? bot fqdn etc.

P.S. Closed issues are not looked at. Could you please reopen/open as a new issue next time? Thanks

[MalarGit]

Looks like a new issue was created already. Closing this one.