Timing information

[po5]

The biggest barrier to improving thumbfast is that we need to attach timing information to our generated thumbnails.
This is important for features like not spamming irrelevant thumbnail updates while the slave is trying to catch up (https://github.com/po5/thumbfast/issues/65#issuecomment-1306543074), optional caching, and for acting as a general purpose thumbnail provider (https://github.com/tomasklaen/uosc/pull/353#issuecomment-1402843403) with reliable thumbnails we know for sure won't randomly crash the player when accessed by 3rd party scripts.

Currently, when we get back a thumbnail we have no way to know which seek caused it, we just know it's the most recent data that came out and doesn't necessarily line up with our very recent seeks.
With the current model of a constantly updated thumbnail file we must wait for it to finish rendering before submitting the next seek, if we want to pinpoint accurate (within a keyframe) times.
Maybe that is fast enough to be viable, if the slave instance waits between frames anyway (untested assumption). We could always spawn more slaves to keep up with seeks, but it quickly gets resource expensive.
Maybe we can figure out a way to guess the seek that's being processed by the slave, but I doubt it.

I've looked into these options for changing the output filename for each thumbnail.

• Updating --o or --ofopts at runtime: doesn't work
• --ofopts=frame_pts with %d: I'd love to use this, but it acts the same way as sequential (mpv bug? or am I using it wrong?) including with --no-correct-pts --fps=24.

It may be possible to encode timing info into the actual thumbnail data, through video filters?

The most reliable solution I can see would be to have a script running on the slave instance akin to #42 that could write down which timestamp it's currently at, or rename the generated file to include a timestamp, on a property update/event.

Do you have other ideas for associating timestamps with thumbnails? Please let me know.

TODO: check if adding --untimed without any other changes improves performance.

[christoph-heinrich]

Running mpv --no-config --ovc=rawvideo --of=image2 --ofopts=frame_pts=1 --o=%04d <some file> does produce files like

0000
0792
1608
2400
3192
4008
4800
5616
6408
7200
8016
8808
9600

But I don't know what unit they are? It's a 29.97 fps file, so maybe they are raw timestamps and need to be scaled by the timebase? I don't know how to extract it and I'll have to leave now, so no time to look it up :wink: but maybe that's already helpful.

[po5]

Weird, I am getting sequential files with that command:

0000
0001
0002
0003
0004
0005

Whatever the issue is, it means it's not a portable solution and can't be used here.

[christoph-heinrich]

Assuming that those are the raw pts, it could be that for your file the timebase is such that those are the actual timestamps. You can look at the pts (and other frame related info) with ffprobe -i <file> -show_frames -select_streams 0. You might have to adjust the selected stream, but without that audio and video frames get interleaved in the output.

Although I've checked that with my file and I can't see how they are connected... Those numbers have to come from somewhere, so what's going on here??? I've also checked a different file, with different fps, got different output filenames from mpv, but still couldn't see how those numbers are related to the ones ffprobe spits out.

Edit: I forgot to mention before, I think we should consider using --ofopts=atomic_writing, as that would prevent us from reading partially written files.