Multiple channel identification when frame is receeived

[Dimigergo]

Hello!

Is there any way to handle if two audio channel are in the stream? It is ok, that the sdp file handles it if the dynamic payload type is different. But when I receive the frame for example RawAACFrame in the rtspClient.FrameReceived += (sender, frame) => method I can't get the channel id so in that case I don't know where it comes from.

Thanks!

[BogdanovKirill]

Hello,

Yes, at the moment you can't understand from which channel you've got data. This should be changed.

[Dimigergo]

Hello!

I have made it locally. Can I get a branch to commit it and you will check it?

Thanks!

[BogdanovKirill]

Hello,

Sure, I need to review it, may be I could add something

[Dimigergo]

Thanks!

Here is my code forked from your master branch: https://github.com/Dimigergo/RtspClientSharp

The main point is the mediaId which identificate the channel from SDP parser.

BTW with this mediaId it could receive channels when SDP "file" contains one audio and video channel with the same payload type (f.e.: 96) described here: https://stackoverflow.com/questions/15091246/same-media-format-for-audio-and-video-on-rtsp In this case, your code parse only the last channel from sdp, because your id is the payload type, which is non unique. If you have problem with my code, I can make any modifications on it, if it is necessary. :)

Thanks in advance!

[BogdanovKirill]

At the moment I have 2 questions about your code: 1) Why do we need "mediaId" when have "trackId" from SDP? It seems that "trackId" is something more natural 2) Is it possible to avoid passing "mediaId" in different subsystems? I need to think about it, may be lambdas and/or delegates could help to insert "mediaId"/"trackId" at finish stage when we are constructing RawFrames

[Dimigergo]

Hello! Sorry for the late answer!

1. I'm wondering, that trackId is ok for us, but the payload type/id as original code isn't. For example the VLC stream server makes video and audio stream as the same payload type, and it is correct way according to the SDP standard. VLC stream SDP is:

   v=0
   o=- 16223588131077641052 16223588131077641052 IN IP4 GDIMI2018
   s=Unnamed
   i=N/A
   c=IN IP4 0.0.0.0
   t=0 0
   a=tool:vlc 3.0.8
   a=recvonly
   a=type:broadcast
   a=charset:UTF-8
   a=control:rtsp://127.0.0.1:8554/valami
   m=video 0 RTP/AVP 96
   b=RR:0
   a=rtpmap:96 H264/90000
   a=fmtp:96 packetization-mode=1;profile-level-id=640028;sprop-parameter-sets=Z2QAKKzTAeAIn5cBagICAoAAAfSAAHUwB4wYnA==,aO68sA==;
   a=control:rtsp://127.0.0.1:8554/valami/trackID=0
   m=audio 0 RTP/AVP 96
   b=RR:0
   a=rtpmap:96 mpeg4-generic/44100/2
   a=fmtp:96 streamtype=5; profile-level-id=15; mode=AAC-hbr; config=1210; SizeLength=13; IndexLength=3; IndexDeltaLength=3; Profile=1;
   a=control:rtsp://127.0.0.1:8554/valami/trackID=1

   When we parse the SDP with the original code only the audio frames are received, because payload type (96) overwrites the _payloadFormatNumberToInfoMap.

I think trackId in the end of the a=control:rtsp://127.0.0.1:8554/valami/trackID=0 is not good, because it can be string or int, or it should streamid or streamId or trackId or trackid, so it can be difficult to parse and the documentation said: "It should take the whole string". But the a:control uri is unique BTW, so it can be unique.

Is it possible, that is it a good idea instead of my mediaId?

2. I absolutely agree, that don't pass mediaId/trackId in all subsystems. For the point 1, it is good to query all SDP information from the "mediaId/trackId". For example FPS in video, samplerate in audio. In this case, generated mediaId is simplier than trackId (track uri), bacause mediaId is generated when media line is parsed, but trackId will be there only in the attributes after media line.

If it is ok and you agree, I try to do it, and send the code to review. ;)

Thanks!

[BogdanovKirill]

Hello,

Thank you for your work and ideas. Yes you are right in general. But may be we should think about new public classe like RtspTrackInfo which will be available in RtspClient class after success connection. This class would contain some useful info from SDP. And also reference to instance of such class should be put in every RawFrame class. That will help to get userful info about tracks and to understand from which track some frame was received.

[Dimigergo]

Ok, I will implement it, and send to you for review. ;)

[BogdanovKirill]

No It doesn't mean that it is your work. I just what to find best solution and then we could implement it

[BogdanovKirill]

Thank you for your help anyway!

[Dimigergo]

Ohh, okay, in this case, I will wait for this implementation. :) Thanks!

[BogdanovKirill]

What do you think about this idea? Do see any minuses or perf issues ?

[Dimigergo]

Temporary in my code that use Dimigergo/RTSPClientSharp with the "mediaId" in upper level I have made it similar to your suggestion, and I haven't got any performance issue with it.

I have made a class which made from hardcoded code. new CMediaInfo(1, 25) It's only contains FPS, because I only need it.

    public class CMediaInfo
    {
        public int MediaId { get; set; }
        public int FPS { get; set; }

        public CMediaInfo(int mediaId, int fps)
        {
            this.MediaId = mediaId;
            this.FPS = fps;
        }

        public override string ToString()
        {
            return "MediaId: " + this.MediaId.ToString() 
                + " FPS: " + this.FPS.ToString();
        }
    }

When the first RawFrame is received with a MediaId I query this CMediaInfo reference from a private list and initialize my code with the saved FPS.

So according to your suggestion if these media info class come from the RtspClient class and filled from SDP the performance is the same as mine with the hardcoded temporary CMediaInfo list.

In my opinion your suggsestion is a good solution. :) I think it is good if every media line creates a new instance with a string list, which contains the attribute lines, and referenced in the frames. In this case I think don't need to parse every line (FPS etc.), it should make in upper level if need.