search

CasparCG / server

CasparCG Server is a Windows and Linux software used to play out professional graphics, audio and video to multiple outputs. It has been in 24/7 broadcast production since 2006. Ready-to-use downloads are available under the Releases tab https://casparcg.com.
GNU General Public License v3.0
901 stars 269 forks source link

MPEG-TS - output delay due to intentionally dropped frames #1374

Open euphoricz opened 3 years ago

euphoricz commented 3 years ago

Expected behaviour

Ignore time gap and continue playing the next frames received

Current behaviour

Output hangs, diag shows frame-time increase, buffer continues to increase if streaming, if using a file buffer is steady, until the expected frame time or count has caught up with the matching first frame in the buffer.

Steps to reproduce

  1. Produce an h264 / h265 video with an mpeg-ts container.
  2. Strip a few seconds out of the middle of the video by stitching i-frames together.
  3. Play in CCG watch output / diag.
  4. When the stitched i-frames are encountered, the diag indicates an up-tick in frame-time at which point the output hangs for the amount of time frames are missing, after which the remaining video will continue to play.

Environment

Additional Info:
This worked in 2.0.7, works correctly in ffplay. This frame dropping behavior is a method of mitigating network congestion and buffer buildup by simply truncating the buffer and stitching i-frames together. I have attempted setting the discontinuity_indicator flag on the ts-packet header but it has no effect.

Let me know if you require an example video file. There is no additional processing taking place when frames are dropped, I'm simply referring to removing frames from the transport stream and stitching i-frames together for a seemless cut.

euphoricz commented 3 years ago

Example diag when stitch occurs in mpeg-ts stream. The underflow is intentional for the purpose of the test, it is caused by the streaming software which limits the send buffer size and introducing a 500 ms sleep every send. Once the buffer size grows to a max allowed size the software removes the excess buffer and "stitches" two i-frames together.

You can see from the diag the stitch coincides with the spike in frame time, at this point CCG stops reading from the buffer for the same amount of time as the amount of frames which were dropped by the streaming software, the output hangs on the frame immediately before the stitch, once the dropped frame time has elapsed the output will continue displaying frames.

image

euphoricz commented 3 years ago

This is an example file which reproduces the behavior.

example.zip

euphoricz commented 3 years ago

In the interim to resolve the issue and ensure the buffer can be managed by splicing / stitching it, the stream is sent through ffmpeg using -probesize 32 -c:v copy -c:a copy to remux it before sending to CCG.

This resolves the problem likely by fixing the timing in the container and allows for seamless splicing with little to no overhead and low latency.

Not the most ideal solution, but it works for now.