UnmixedAudioBuffers is null

[DjTeo]

Describe the issue I am working on the PolicyRecordingBot sample, I have set new AudioSocketSettings { StreamDirections = StreamDirection.Recvonly, SupportedAudioFormat = AudioFormat.Pcm16K, ReceiveUnmixedMeetingAudio = true, } and I want to record unmixed audio of a meeting in a file (.wav), I make http call to /JoinCall as in AudioVideoPlaybackBot sample. On OnAudioMediaReceived event the e.Buffer.UnmixedAudioBuffers is always null. I tried to save e.Buffer.Data to .wav file and it seems that unmixed audio goes there. I tried to listening it and is a fast forward audio with a lot of cuts, like if in every audio buffer frame its changed participants' audio. I didn't have trouble recording mixed audio.

Expected behavior I was expected the e.Buffer.UnmixedAudioBuffers to have value like the documentation says.

Graph SDK :

• Don't know where this is.
• Microsfot.skype.bots.media nuget is 1.19.0.25-alpha
• Microsoft.Graph.Communications.* nugets are 1.2.0.850

Call ID 457f835d-5d53-4aa6-ad2d-a1627f84432a

Logs

2020-06-29T11:13:51.5050373Z Info: GraphAuthClient.cs:100 SendHttpRequestAsync
ScenarioId: 757aee3b-21cb-4df9-8194-ea5663c25e5e
AppId: 0980ff84-61c1-43f4-927a-4a7b31ae5dd2
AppName: Sample.PolicyRecordingBot.FrontEnd
TransactionDirection: Outgoing
TraceType: HttpRequest
request: POST https://graph.microsoft.com/v1.0/communications/calls
headers:
  Scenario-Id: 757aee3b-21cb-4df9-8194-ea5663c25e5e
  Client-Request-Id: 6eecc24c-6bf2-44d0-a66b-23dde5763ac5
  User-Agent: GraphCommunicationsClient-Sample.PolicyRecordingBot.FrontEnd/1.2.0.850
  Authorization: (redacted)
  Content-Type: application/json; charset=utf-8
{
  "@odata.type": "#microsoft.graph.call",
  "direction": "outgoing",
  "callbackUri": "https://teambot.abc:10101/api/calling/notification",
  "source": {
    "@odata.type": "#microsoft.graph.participantInfo",
    "identity": {
      "@odata.type": "#microsoft.graph.identitySet",
      "application": {
        "@odata.type": "#microsoft.graph.identity",
        "displayName": "(Identity: qwer)",
        "id": "example-guid"
      },
      "guest": {
        "@odata.type": "#microsoft.graph.identity",
        "displayName": "(Identity: asdf)",
        "id": "example-guid"
      }
    }
  },
  "requestedModalities": [
    "audio"
  ],
  "mediaConfig": {
    "@odata.type": "#microsoft.graph.appHostedMediaConfig",
    "blob": "{\"mpUri\":\"net.tcp://teambot.abc:20101/MediaProcessor\",\"audioRenderContexts\":[\"73cfbf4f-ae42-47af-a741-6b1b8ddf7185\"],\"videoRenderContexts\":[],\"audioSourceContexts\":[null],\"videoSourceContexts\":[],\"dataRenderContexts\":null,\"dataSourceContexts\":null,\"supportedAudioFormat\":\"Pcm16K\",\"videoSinkEncodingFormats\":[],\"mpMediaSessionId\":\"93aeb344-2196-4a16-97a9-e3d2ab6aabd4\",\"regionAffinity\":null,\"skypeMediaBotsVersion\":\"1.19.0.0025\",\"mediaStackVersion\":\"2020.18.1.16\",\"mpVersion\":\"7.2.0.9053\",\"callId\":\"457f835d-5d53-4aa6-ad2d-a1627f84432a\",\"receiveUnmixedMeetingAudio\":true}"
  },
  "chatInfo": {
    "@odata.type": "#microsoft.graph.chatInfo",
    "threadId": "19:meeting_Yjg2NDM2OTMtNGEzYy00OWRhLTgyMjAtYzE2MDNlMzcxM2Qz@thread.v2",
    "messageId": "0"
  },
  "meetingInfo": {
    "@odata.type": "#microsoft.graph.organizerMeetingInfo",
    "organizer": {
      "@odata.type": "#microsoft.graph.identitySet",
      "user": {
        "@odata.type": "#microsoft.graph.identity",
        "id": "example-guid",
        "tenantId": "example-guid"
      }
    }
  },
  "tenantId": "example-guid",
  "callOptions": {
    "@odata.type": "#microsoft.graph.outgoingCallOptions",
    "allowGuestToBypassLobby": null
  }
}

[adityaramgopal]

It's because it gets considered as an anonymous user join. Assuming you've taken parts of the AudioPlaybackBot sample, can you remove this code snippet and try again?

            if (!string.IsNullOrWhiteSpace(joinCallBody.DisplayName))
            {
                // Teams client does not allow changing of ones own display name.
                // If display name is specified, we join as anonymous (guest) user
                // with the specified display name.  This will put bot into lobby
                // unless lobby bypass is disabled.
                joinParams.GuestIdentity = new Identity
                {
                    Id = Guid.NewGuid().ToString(),
                    DisplayName = joinCallBody.DisplayName,
                };
            }

[DjTeo]

Thank you very much, it worked! But when nobody is speaking the e.Buffer.UnmixedAudioBuffers is null instead of an empty array, is this intended?

[adityaramgopal]

Yes, this is by design. When no one is speaking e.Buffer.UnmixedAudioBuffers will be null. It will appear as a silent "mixed buffer" type of a packet. So IsSilence property on the AudioMediaBuffer will be true, the Data property will point to valid (silence) data, and the Length will be the length of the silence buffer. We recommend you write this buffer to all the individual .wav files you have for each speaker so that it accounts for the silence in between the speech.

[DjTeo]

Thank you very much again. That explains a lot.