search

sipsorcery-org / sipsorcery

A WebRTC, SIP and VoIP library for C# and .NET. Designed for real-time communications apps.
https://sipsorcery-org.github.io/sipsorcery
Other
1.47k stars 442 forks source link

Format Negotiation Fails #981

Closed SipSorceryFan closed 9 months ago

SipSorceryFan commented 1 year ago

We're having trouble negotiating the video format between our application using SIPSorcery and another 3rd-party device. Using the https://github.com/sipsorcery-org/SIPSorceryMedia.FFmpeg/blob/master/src/FFmpegVideoEndPoint.cs class we report support for both VP8 and H264 while the mentioned 3rd-party device simply reports H264. We expect then our application to use H264 yet the received VideoFormat instance supplied to the FFmpegVideoEndPoint.SetVideoSourceFormat() method is always VP8.

I've created a very simple self-contained demonstration program depending only on the main SIPSorcery package available here: https://github.com/SipSorceryFan/SIPSorceryVideoNegotiationTest

This demonstration contains two IVideoSource implementations, one which mimics exactly the supported formats from FFmpegVideoEndPoint and one which mimics the 3rd-party device reporting only H264. These IVideoSource implementations simply print the reported VideoFormat inside SetVideoSourceFormat() to the console for easy viewing. A caller user-agent is created with the multi-format video source and a recipient user-agent is created with the single-format video source to mimic our environment.

The expected output of running the above is:

SingleFormatVideoSource Video Source Format Id: 100 MultiFormatVideoSource Video Source Format Id: 100

While the actual output is below, note here the format id of "96" indicating VP8 which is what our actual setup is receiving:

SingleFormatVideoSource Video Source Format Id: 100 MultiFormatVideoSource Video Source Format Id: 96

From reviewing the source code it seems a commented-out line may be the fix, specifically line 1032 of RTPSession: https://github.com/sipsorcery-org/sipsorcery/blob/master/src/net/RTP/RTPSession.cs#L1032

Admittedly I'm quite new to SIP (and SIPSorcery) so this may all be user-error but simply un-commenting this line resolves our specific setup and correctly sets our local video source format to H264: currentMediaStream.LocalTrack.Capabilities = capabilities;

I was not able to track down any other configuration settings related to negotiation but if this is user-error I'm happy to do some more reading.

ChristopheI commented 1 year ago

Thanks for the input. I'll try it when I have some time