Support Timestamp Discontinuity

[lotharkript]

Some container allows Timestamp discontinuity, like Mpeg 2 Transport stream.

If you get a file where there are 2 packets with the same timestamp (for example in case of concat twice the same TS file), the order of the frame index returns from FFMS_GetFrameInfo() is not correct.

How to reproduce: create a small TS file /ffmpeg -f rawvideo -s 10x10 -i /dev/zero -t 1 /tmp/test.ts cp /tmp/test.ts /tmp/test2.ts cat /tmp/test2.ts >> /tmp/test.ts

Then index the file /tmp/test.ts and loop over the video track using FFMS_GetFrameInfo() you will get Frame[0] = 129600 Frame[1] = 129600 <-- this frame seems to be from the second part of the video. Frame[2] = 133200 Frame[3] = 133200 <-- dito Frame[4] = 136800 Frame[5] = 136800 <--dito

[myrsloik]

Not supported use case. Use d2vsource instead for all your mpeg needs.

[tgoyne]

Seems conceptually simple to fix as long as the concat point can be reliably distinguished from frames just being in a funny order (which is easy if the DTS is reliable). Just need to synthesize fixed PTSes for the post-concat frames so that they aren't sorted incorrect and then always seek by byte pos rather than pts for such files.

[dwbuiten]

@tgoyne I assume this would be some separate field? I am interested in being able to access the original PTS still, if you were going to go that way. It's fine if it's API-only, which would make sense. I can submit code to do so. It's not useful info for AVS/VS plugin users, who'd benefit from just a working video with sane-ish timestamps.

Currently, I am using my own ffindex parser for various things. I'd love to ditch it.

(Also I would not use d2vsource for this; it's basically the worst code I've ever written.)

[dwbuiten]

@tgoyne Do you have any particular inclination towards API design for this?

There's a couple ways we could do this, bearing in mind it's not really "possible" to always detect the concat point, or even if there is one, or if it's just wraparound, etc.

Strategy 1

• Add an some sort of order, or original frame number field to the index

• An API like:

  const FFMS_Frame * FFMS_GetFrameInFileOrder(FFMS_VideoSource *V, int n, FFMS_ErrorInfo *ErrorInfo);
  const FFMS_FrameInfo *FFMS_GetFrameInfoFileOrder(FFMS_Track *T, int Frame);

Ignoring my poor naming choices, this strikes me as still fairly ugly, and obviously no analogue to FFMS_GetFrameByTime can be added.

Strategy 2

Add an "original PTS" field index and either expose it as a field in the public frame struct (ABI break) or add a getter function on top of the frame or some struct (ugly). This also presents a few problems for the user, in that if they use the only the API (and not their own lib, like me), they have to loop over all frames using FFMS_GetFrameInfo and create their own frame-to-original-frame-order map. Also pretty ugly / not great, to me.

Strategy 3

Don't sort at all, and let the API user handle crappy timestamps, and perhaps provide a "unbreak timestamp" function like:

int64_t FFMS_SanitizeTimestamp(FFMS_Track *T, int64_t OldTimestamp);

or some batch processing function that fills or returns an array of timestamps. This is, yet again, fairly ugly, and would require renaming FFMS_GetFrame and pals, because it is an behavior change.

All three of these would also entail changing the linear-decoding-detection code to check against original order and/or byte pos, keeping in mind some formats may not necessarily order samples in monotonically increasing byte order (looking at you, MP4). I also haven't thought much about audio, since I do not personally use the API for audio, so input is needed here.

So, pretty much fugly all around. @myrsloik and @lotharkript - your ideas and comments are welcome here too - perhaps you can think of something less ugly than I have.

[lotharkript]

From what i understand, only a few container allows timestamp discontinuity, aka Mpeg 2 TS. The spec defined what it means. Could we not during the indexer convert the timestamp discontinuity to a new PTS/DTS value so the frame will still be in the correct order, but in the core of the indexer, remember the offset we added to the PTS/DTS to reach that frame? I have a question for the dev: Do we care to return the PTS/DTS from the file to the user, or just the computed one form the indexer?

[dwbuiten]

Could we not during the indexer convert the timestamp discontinuity to a new PTS/DTS value so the frame will still be in the correct order, but in the core of the indexer, remember the offset we added to the PTS/DTS to reach that frame?

This depends if libavformat will even be able to seek by timestamp in such a case (it uses a binary search for some formats, at least). Not sure if keeping the offset around will be useful if it is not planned to expose the original PTS/DTS to the user, since we'd likely have to seek by byte position.

[dwbuiten]

This should be merged into the ffms2000 branch now.

@lotharkript Feel free to re-open if there's a problem with the implemenation. Closing for now.