scottlamb / retina

High-level RTSP multimedia streaming library, in Rust
https://crates.io/crates/retina
Apache License 2.0
244 stars 48 forks source link

add dts extractor #61 #81

Open Curid opened 1 year ago

Curid commented 1 year ago

This is a rust port of the gortsplib dts extractor. I don't really understand how it works, but my test stream doesn't stutter anymore.

I know you're busy, I don't expect this to be reviewed anytime soon.

scottlamb commented 1 year ago

First, thank you!

I don't own any cameras that produce B frames, so...

and...

I don't really understand how it works, but my test stream doesn't stutter anymore.

I don't really either and need to make another pass through to develop that understanding and compare this technique with gstreamer's h264timestamper [edit: code here]. Haven't looked closely enough yet to see if gstreamer and rtsp-simple-server (now called mediamtx apparently) are doing the same thing (and, if not, which is better). But I made one pass through and left some comments anyway.

Curid commented 1 year ago

The gstreamer implementation seems to add latency. I'm not sure if it's just some initial delay like with the MediaMTX implementation but otherwise that's a problem.

In order to determine the DTS of each frame, this element may need to hold back a few frames in case the codec data indicates that frame reordering is allowed for the given stream. That means this element may introduce additional latency for the DTS decision.

Curid commented 6 months ago

Found an explanation for how this algorithm works: https://github.com/bluenviron/mediamtx/issues/1002#issuecomment-2115930927

Curid commented 4 months ago

Would it be better to first add the dts extractor as separate module that people can use manually before integrating it into the Depacketizer?