microsoftgraph / microsoft-graph-comms-samples

Microsoft Graph Communications Samples
MIT License
208 stars 225 forks source link

[AKS] Bot don't connect to a long-running pod in the cluster #464

Closed adoronin closed 3 years ago

adoronin commented 3 years ago

Hi all, We have an issue with the bot deployed to the Azure Kubernetes Cluster. All was ok, the pod was alive for 48 days. But then the bot began to not connect to a meeting with an error: "code": 500, "message": "MediaControllerConversationAddParticipantFailed", "subcode": 7112

Restarting the pod helped but we need to discover what was causing the issue and how we can prevent that in the future. I assume that was caused by the re-issuing certificate by the cert-manager. All settings relating to cert-manager left the same as in the AKS sample. Currently I'm trying to reproduce that issue but with no success yet. My tries to simulate re-issuing the certificate has not given a result yet - the bot successfully connected to a meeting event after I delete the secret & certificate and cert-manager created that again. We need to prevent that issue in the future due to the bot is one of the key functionality of our project.

Graph SDK (please complete the following information): Microsoft.Skype.Bots.Media v1.19.0.25-alpha Microsoft.Graph.Communications.Calls.Media v1.2.0.1702

Call ID call ID: c01f6100-639c-4f1e-8813-de47ac720313 UTC time: 2021-06-08T19:21:34.9655405Z

Logs [19:21:34 INF] $>2021-06-08T19:21:34.9655405Z Info: CommunicationsClientExtensions.cs:151 LogAndCreateResponse ScenarioId: 0a13ff82-7ec6-431e-804a-4fa899288d6c AppId: 8cc39710-03f6-446a-b892-546c57188999 AppName: Interprefy TransactionDirection: Incoming TraceType: HttpRequest request: POST https://teams-bot-0.teams-bot.teams-bot.svc.cluster.local:9441/0/api/calling headers: X-Microsoft-Skype-Chain-ID: 81888311-4cbd-47a5-b7a7-3128e2a07ba9 Scenario-Id: 0a13ff82-7ec6-431e-804a-4fa899288d6c X-Microsoft-Skype-Message-ID: 33ec43ae-79fc-4297-a828-5e6099fce92b Connection: close Accept: application/json Authorization: (redacted) Host: teams-bot-0.teams-bot.teams-bot.svc.cluster.local:9441 User-Agent: Microsoft-Skype/3.0,(Calling/1.0) Content-Length: 977 Content-Type: application/json { "@odata.type": "#microsoft.graph.commsNotifications", "value": [ { "@odata.type": "#microsoft.graph.commsNotification", "changeType": "deleted", "resourceUrl": "/communications/calls/c01f6100-639c-4f1e-8813-de47ac720313", "resource": "/app/calls/c01f6100-639c-4f1e-8813-de47ac720313", "resourceData": { "@odata.type": "#microsoft.graph.call", "callChainId": "81888311-4cbd-47a5-b7a7-3128e2a07ba9", "chatInfo": { "@odata.type": "#microsoft.graph.chatInfo", "messageId": "0", "threadId": "19:meeting_YTIyNzlhYjUtNzFmNi00OTZjLTg0OWItZDY5YjgwMTM2OWRj@thread.v2" }, "meetingInfo": { "@odata.type": "#microsoft.graph.organizerMeetingInfo", "organizer": { "@odata.type": "#microsoft.graph.identitySet", "user": { "@odata.type": "#microsoft.graph.identity", "id": "d3705a81-d813-48e6-b7af-5ea9cc4c356d", "tenantId": "1a33b323-4724-41a6-aef3-9645e3e92b3d" } } }, "resultInfo": { "@odata.type": "#microsoft.graph.resultInfo", "code": 500, "message": "MediaControllerConversationAddParticipantFailed", "subcode": 7112 }, "state": "terminated", "tenantId": "1a33b323-4724-41a6-aef3-9645e3e92b3d", "id": "c01f6100-639c-4f1e-8813-de47ac720313" } } ] } $>2021-06-08T19:21:34.9655405Z Info: CommunicationsClientExtensions.cs:151 LogAndCreateResponse ScenarioId: 0a13ff82-7ec6-431e-804a-4fa899288d6c AppId: 8cc39710-03f6-446a-b892-546c57188999 AppName: Interprefy TransactionDirection: Incoming TraceType: HttpRequest request: POST https://teams-bot-0.teams-bot.teams-bot.svc.cluster.local:9441/0/api/calling headers: X-Microsoft-Skype-Chain-ID: 81888311-4cbd-47a5-b7a7-3128e2a07ba9 Scenario-Id: 0a13ff82-7ec6-431e-804a-4fa899288d6c X-Microsoft-Skype-Message-ID: 33ec43ae-79fc-4297-a828-5e6099fce92b Connection: close Accept: application/json Authorization: (redacted) Host: teams-bot-0.teams-bot.teams-bot.svc.cluster.local:9441 User-Agent: Microsoft-Skype/3.0,(Calling/1.0) Content-Length: 977 Content-Type: application/json { "@odata.type": "#microsoft.graph.commsNotifications", "value": [ { "@odata.type": "#microsoft.graph.commsNotification", "changeType": "deleted", "resourceUrl": "/communications/calls/c01f6100-639c-4f1e-8813-de47ac720313", "resource": "/app/calls/c01f6100-639c-4f1e-8813-de47ac720313", "resourceData": { "@odata.type": "#microsoft.graph.call", "callChainId": "81888311-4cbd-47a5-b7a7-3128e2a07ba9", "chatInfo": { "@odata.type": "#microsoft.graph.chatInfo", "messageId": "0", "threadId": "19:meeting_YTIyNzlhYjUtNzFmNi00OTZjLTg0OWItZDY5YjgwMTM2OWRj@thread.v2" }, "meetingInfo": { "@odata.type": "#microsoft.graph.organizerMeetingInfo", "organizer": { "@odata.type": "#microsoft.graph.identitySet", "user": { "@odata.type": "#microsoft.graph.identity", "id": "d3705a81-d813-48e6-b7af-5ea9cc4c356d", "tenantId": "1a33b323-4724-41a6-aef3-9645e3e92b3d" } } }, "resultInfo": { "@odata.type": "#microsoft.graph.resultInfo", "code": 500, "message": "MediaControllerConversationAddParticipantFailed", "subcode": 7112 }, "state": "terminated", "tenantId": "1a33b323-4724-41a6-aef3-9645e3e92b3d", "id": "c01f6100-639c-4f1e-8813-de47ac720313" } } ] }

ssulzer commented 3 years ago

@adoronin "MediaControllerConversationAddParticipantFailed" errors indicate that we could not establish the necessary control channel between our bot media controller (running in the Microsoft Teams bot platform services) and the bot media SDK running inside the bot application. During set up of that control channel the bot's server certificate is examined.

In this particular call failure, the bot's server certificate (with thumbprint beginning with 712484...) was expired. If you provision a new server cert for the bot you need to restart the bot process so the new cert is picked up and given to the bot media platform on startup.

adoronin commented 3 years ago

It's clear now. Thank you @ssulzer!