livepeer / verification-classifier

Metrics-based Verification Classifier
MIT License
8 stars 7 forks source link

Investigate potential sources of frame/timestamp misalignment between source and rendition in LPMS #96

Closed yondonfu closed 4 years ago

yondonfu commented 4 years ago

If you apply ffmpeg's FPS filter to a source video so that it matches the FPS filter behavior of the LPMS transcoder should we expect frame/timestamp alignment between the source and rendition? Or are there other things in the LPMS transcoder that can create frame/timestamp misalignment.

j0sh commented 4 years ago

should we expect frame/timestamp alignment between the source and rendition?

More or less. One thing that could affect this is if the target container changes, then the underlying time base would change (the resolution or number of "ticks" of the clock within a given interval, typically 1 second), which rescales the resulting timestamps. For example, in mpegts, the timebase is 90khz, while in FLV (RTMP) it's 1khz. In MP4 it can vary - I have a BBB sample here with a 48khz timebase for audio and 30khz for video.

While we only work on mpegts at the moment, we could some day go from mpegts to mp4 to mkv/webm, so the source time base should be rescaled to match the target, even though that's typically a no-op right now.

Or are there other things in the LPMS transcoder that can create frame/timestamp misalignment.

LPMS itself doesn't do much else, but the ffmpeg CLI will try to do a few other things to sync up audio / video, which is important to be aware of if the CLI is being used. I think supplying the -vsync 0 option to the ffmpeg CLI should match LPMS behavior most closely. (We might also have relied on -copyts at some point but I think we're okay without it now.)

yondonfu commented 4 years ago

Closed by #111