search

microsoftgraph / microsoft-graph-comms-samples

Microsoft Graph Communications Samples
MIT License
206 stars 224 forks source link

Giving a bot the best chance to receive HD video #305

Open paulnearney opened 4 years ago

paulnearney commented 4 years ago

For my use case, it's very important that my bot receives HD video from a meeting - at least 720p.

I've been testing today, and I've noticed that the video resolution received by a bot (hosted on an Azure VM) varies according to how many participants it's subscribed to. I'm requesting 720p video, and I can see that I receive that when i'm only subscribed to a single participant's feed, but as I subscribe to more participants, the resolution of the video received for each participant drops - 2 participants, each is received at 540x960, and this drops further when adding more participants. It doesn't seem to matter whether the participants are in the same meeting, or in different meetings - the same behaviour seems to happen.

It would be good to know where and why this decision is being made - e.g. is it Teams deciding to lower the resolution based on the fact that's it's sending multiple streams to the same IP address, or is it e.g. bandwidth restriction on the VM the bot is running on. What I want to get to is a deployment architecture that gives my bots the best chance of receiving HD video (provided it's being sent by the meeting participant, obviously).

There's some helpful info from @ssulzer in #151, but this doesn't completely answer the question.

paulnearney commented 3 years ago

Any update on this at all? @ksikorsk @ssulzer

paulnearney commented 3 years ago

Is there any update on this please?

ssulzer commented 3 years ago

What kind of Azure VM is your bot running on? e.g., Standard_D2v2?

ssulzer commented 3 years ago

First thing to check is whether the VM isn't powerful enough to handle multiple 720p decodings. Are you receiving video in NV12 or H264 format?

paulnearney commented 3 years ago

The VM is Standard_B2s, which I appreciate is fairly small - it was a while ago, but I don't recall seeing any obvious bottlenecks on the machine itself with more than one call. I'm receiving NV12 as I need to be able to pick complete frames out at regular intervals

ssulzer commented 3 years ago

The "burstable" B-series VMs are not suited for real-time media processing. Please try an Fsv2-series VM (Standard_F4s_v2 or larger), or a Dv3-series (Standard_D4_v3 or larger).