livepeer / lpms

Livepeer media server
MIT License
282 stars 71 forks source link

b-frame stream: start pts correction #363

Closed AlexKordic closed 1 month ago

AlexKordic commented 1 year ago

Fixing scenario where input stream contains B-frames. Because we process video segments individually, often multiple transcoders process different segments in parallel, several-frame-long gap is produced on segment boundary.

Here is input segment timing example:

[PTS 0.000s] [DTS 0.000s] [Payload 278876]
[PTS 0.066s] [DTS 0.033s] [Payload 206302]
[PTS 0.033s] [DTS 0.066s] [Payload 144201]
// ...
[PTS 5.799s] [DTS 5.766s] [Payload 245908]
[PTS 5.767s] [DTS 5.800s] [Payload 148152]
[PTS 5.866s] [DTS 5.833s] [Payload 199768]
[PTS 5.833s] [DTS 5.866s] [Payload 152500]
[PTS 5.933s] [DTS 5.900s] [Payload 200964]
[PTS 5.900s] [DTS 5.933s] [Payload 160400]
// ------- Segment boundary -------
[PTS 5.999s] [DTS 5.966s] [Payload 255143]  // this is IDR frame
[PTS 5.967s] [DTS 6.000s] [Payload 152224]  // this one is displayed before IDR, referencing portions of IDR frame
[PTS 6.066s] [DTS 6.033s] [Payload 204143]
[PTS 6.033s] [DTS 6.066s] [Payload 157830]
[PTS 6.133s] [DTS 6.100s] [Payload 188788]
[PTS 6.100s] [DTS 6.133s] [Payload 154605]
// ...

This fix adjusts timestamps so first frame of output stream starts from 5.967s instead from 6.000s.

How to test

cd ffmpeg go test . -v -count 1 -tags nvidia -run TestTranscoder_offsetStartPTS

ffprobe should be on the PATH for unit test to work.

j0sh commented 1 month ago

Closing this since transcoder2 was removed in https://github.com/livepeer/lpms/pull/410

also I am not sure about this sample - this is HEVC with a leading CRA slice (not an IDR), which is followed by a frame that references the previous segment. As such this segment is not a closed GOP and not decodable on its own. That points to a problem with the segmenter, since for our purposes we need segments that can be decoded independently.