access raw RTP packets (headers and all)

[RustPanda]

Hello. How to Get raw Rtp Package? I want to redirect rtp packages to gstreamer rtph264depay.

[RustPanda]

webrtc-rs example:

let mut buf = [0u8; 1500];
while let Ok((size, _)) = track.read(&mut buf).await {
    let mut buffer = gst::Buffer::with_size(size)
        .map_err(|err| tracing::error!("{err}"))
        .unwrap();
    {
        let buffer = buffer.get_mut().unwrap();
        buffer
            .copy_from_slice(0, &buf[..size])
            .map_err(|err| tracing::error!("{err}"))
            .unwrap();
    }

    appsrc
        .push_buffer(buffer)
        .map_err(|err| tracing::error!("{err}"))
        .unwrap();
}

[scottlamb]

"Package" = "packet", right?

retina::client::Session implements futures::Stream<Item = Result<retina::client::PacketItem, retina::Error>>, so roughly:

use futures::StreamExt; // for StreamExt::next

while let Some(pkt) = session.next().await {
    let pkt = pkt.unwrap();
    match pkt {
        PacketItem::RtpPacket(p) => {
            // see p.timestamp, p.payload, etc.
        },
        _ => {} // RTCP sender report; additional payload types in the future
    }
}

[RustPanda]

@scottlamb yes, packet. I want to get slice, raw bytes:

let mut buf = [0u8; 1500]; <--- my buffer for rtp packet.
while let Ok((size, _)) = track.read(&mut buf).await { <--- I receive a packet
...

PacketItem::RtpPacket(p) != &buf[..size] :)

I can not send PacketItem::RtpPacket(p) to rtph264depay I need a non-deserialized package to integrate gstreamre

[scottlamb]

Oh, the full unparsed RTP packet including the headers? That's not currently available; the non-payload parts are trimmed away here:

https://github.com/scottlamb/retina/blob/375754a88fb12bc9d7a59153393b14f5bae6f4e6/src/client/rtp.rs#L203-L204

There's no reason we couldn't keep it around, except that we currently expose the Bytes of the payload, and I'd prefer not to have two Bytes (which would both bloat the struct and require an extra refcount). In the next breaking API change I'm likely to get rid of the exposed payload Bytes anyway in favor of borrowing a &[u8] or similar. (See #47.) Then making the full RTP packet available would be no problem.

Out of curiosity, is there a particular reason you want to use gstreamer's rtph264depay rather than Retina's H.264 depacketization code? I'm not too familiar with gstreamer but I have to imagine they also provide a way to pass in the already-assembled access units as in retina::codec::VideoFrame.

[RustPanda]

@scottlamb i tried to use gstreamer-rtp to send rtp payload to gst-pipeline but it doesn't work.

More convenient to send raw rtp to Gstreamer and webrtc-rs.

[scottlamb]

I mean skipping the rtph264depay step, instead feeding output from Retina's Demuxer (which does roughly the same thing) directly into whatever you'd otherwise feed rtph264depay's output into. I imagine it's possible, but I'm not sure how due to my inexperience with gstreamer.

But anyway, I'm working on #47 and think with those API changes, it'll be pretty easy to support getting the full raw RTP packets.

[scottlamb]

I just pushed the next branch with a bunch of API changes, including one that lets you get the full raw packet as requested here. I'm not done with all the breaking changes, but if you enjoy the bleeding edge, please check it out.

[RustPanda]

@scottlamb thank you very much

[RustPanda]

@scottlamb, i managed to get a stable connection. Here is the code I have been playing with: RTSPlay

[scottlamb]

Cool! Glad it worked, and thanks for sharing that code. Learning more about GStreamer has been on my todo list. If I get a chance, I might play around with your code and see if I can remove the rtph264depay as I suggested above.

From what little I've seen, I really like GStreamer's pipeline model. Seems like overkill for Retina by itself, but maybe someday someone will write a larger media framework in pure Rust...