RtpStream does not exclude RTP package when the payload type of RTP package mismatches with payload number in describe（SDP）

[xingshuangs]

Describe the bug I 'm not sure if this is a bug, as this is first time I commit the issue. In my case, I need to remote control cameras, just rotate it. When I rotate it, the timestamp of RawFrame in feedback will be discontinuous, so I think the frame is alse not discontinuous. When I do not rotate it, the frame will be continuous.

Why?

I find that the payload type is 105 and the sequence number adds 1 normally in every rtp package when I do not remote rotate, the payload type will be 50 and the sequence number will not be changed in rtp package as I rotate. In the function 'ProcessImmediately' of RtpSteam class, it does not exclude the rtp package mismatches with payload number in SDP, and it invokes ‘_mediaPayloadParser.ResetState()’ immediately when the ‘SeqNumber‘ offset is not equal to 1.

 SyncSourceId = rtpPacket.SyncSourceId;
            if (!_isFirstPacket)
            {
                int delta = (ushort) (rtpPacket.SeqNumber - _previousSeqNumber);

                if (delta != 1)
                {
                    int lostCount = delta - 1;

                    if (lostCount == -1)
                        lostCount = ushort.MaxValue;

                    CumulativePacketLost += (uint) lostCount;

                    if (CumulativePacketLost > 0x7FFFFF)
                        CumulativePacketLost = 0x7FFFFF;

                    PacketsLostSinceLastReset += lostCount;

                    _mediaPayloadParser.ResetState();
                }

                if (rtpPacket.SeqNumber < HighestSequenceNumberReceived)
                    ++SequenceCycles;

                _samplesSum += rtpPacket.Timestamp - _previousTimestamp;
            }

To Change Steps to change the behavior:

1. In 'PayloadFormatInfo' class, I add 'PayloadFormatNumber' property, and add 'PayloadFormatNumber' to 'PayloadFormatInfo' object when 'ParseMediaLine' .

   private class PayloadFormatInfo
       {
           public string TrackName { get; set; }
           public CodecInfo CodecInfo { get; set; }
           public int SamplesFrequency { get; set; }
           public int PayloadFormatNumber { get; set; }
   
           public PayloadFormatInfo(CodecInfo codecInfo, int samplesFrequency,int payloadFormatNumber)
           {
               CodecInfo = codecInfo;
               SamplesFrequency = samplesFrequency;
               PayloadFormatNumber = payloadFormatNumber;
           }
       }

   private void ParseMediaLine(string line)
       {
           string[] tokens = line.Split(new[] { ' ' }, StringSplitOptions.RemoveEmptyEntries);
   
           if (tokens.Length < 4)
           {
               _lastParsedFormatInfo = null;
               return;
           }
   
           string payloadFormat = tokens[3];
   
           if (!int.TryParse(payloadFormat, out int payloadFormatNumber))
           {
               _lastParsedFormatInfo = null;
               return;
           }
   
           CodecInfo codecInfo = TryCreateCodecInfo(payloadFormatNumber);
           int samplesFrequency = GetSamplesFrequencyFromPayloadType(payloadFormatNumber);
   
           _lastParsedFormatInfo = new PayloadFormatInfo(codecInfo, samplesFrequency, payloadFormatNumber);
           _payloadFormatNumberToInfoMap[payloadFormatNumber] = _lastParsedFormatInfo;
       }

2. Add ‘_payloadFormatNumber’ field in ‘RtpStream’ class

   
   class RtpStream : ITransportStream, IRtpStatisticsProvider
   {
       private readonly IRtpSequenceAssembler _rtpSequenceAssembler;
       private readonly IMediaPayloadParser _mediaPayloadParser;
       private readonly int _samplesFrequency;
       private readonly int _payloadFormatNumber;

.... ....

public RtpStream(IMediaPayloadParser mediaPayloadParser, int samplesFrequency, IRtpSequenceAssembler rtpSequenceAssembler = null, int payloadFormatNumber = -1) { _mediaPayloadParser = mediaPayloadParser ?? throw new ArgumentNullException(nameof(mediaPayloadParser)); _samplesFrequency = samplesFrequency;

        if (rtpSequenceAssembler != null)
        {
            _rtpSequenceAssembler = rtpSequenceAssembler;
            _rtpSequenceAssembler.PacketPassed += ProcessImmediately;
        }
        _payloadFormatNumber = payloadFormatNumber;
    }

add ‘track.PayloadFormatNumber’ to ‘RtpStream’ object in the function 'SetupTrackAsync' of 'RtspClientInternal' class,

var rtpStream = new RtpStream(mediaPayloadParser, track.SamplesFrequency, rtpSequenceAssembler,track.PayloadFormatNumber); _streamsMap.Add(rtpChannelNumber, rtpStream);

3.  Add filter code in the function 'ProcessImmediately' of 'RtpStream' class .

private void ProcessImmediately(ref RtpPacket rtpPacket) { SyncSourceId = rtpPacket.SyncSourceId;

        if (_payloadFormatNumber > 0 && _payloadFormatNumber != rtpPacket.PayloadType) return;

        if (!_isFirstPacket)
        {
            int delta = (ushort) (rtpPacket.SeqNumber - _previousSeqNumber);

            if (delta != 1)
            {
                int lostCount = delta - 1;

                if (lostCount == -1)
                    lostCount = ushort.MaxValue;

                CumulativePacketLost += (uint) lostCount;

                if (CumulativePacketLost > 0x7FFFFF)
                    CumulativePacketLost = 0x7FFFFF;

                PacketsLostSinceLastReset += lostCount;

                _mediaPayloadParser.ResetState();
            }

            if (rtpPacket.SeqNumber < HighestSequenceNumberReceived)
                ++SequenceCycles;

            _samplesSum += rtpPacket.Timestamp - _previousTimestamp;
        }

4. that‘s all.

**Additional context**
Looking forward to your reply.