search

scottlamb / retina

High-level RTSP multimedia streaming library, in Rust
https://crates.io/crates/retina
Apache License 2.0
218 stars 46 forks source link

add dts extractor #61 #81

Open Curid opened 1 year ago

Curid commented 1 year ago

This is a rust port of the gortsplib dts extractor. I don't really understand how it works, but my test stream doesn't stutter anymore.

I know you're busy, I don't expect this to be reviewed anytime soon.

scottlamb commented 11 months ago

First, thank you!

I don't own any cameras that produce B frames, so...

and...

I don't really understand how it works, but my test stream doesn't stutter anymore.

I don't really either and need to make another pass through to develop that understanding and compare this technique with gstreamer's h264timestamper [edit: code here]. Haven't looked closely enough yet to see if gstreamer and rtsp-simple-server (now called mediamtx apparently) are doing the same thing (and, if not, which is better). But I made one pass through and left some comments anyway.

Curid commented 11 months ago

The gstreamer implementation seems to add latency. I'm not sure if it's just some initial delay like with the MediaMTX implementation but otherwise that's a problem.

In order to determine the DTS of each frame, this element may need to hold back a few frames in case the codec data indicates that frame reordering is allowed for the given stream. That means this element may introduce additional latency for the DTS decision.

Curid commented 1 month ago

Found an explanation for how this algorithm works: https://github.com/bluenviron/mediamtx/issues/1002#issuecomment-2115930927