search

arthurhammer / FrameGrabber

iOS app to extract video frames as images
https://framegrabberapp.com
Other
361 stars 88 forks source link

Frame counter instead of centiseconds #2

Closed velihukka closed 3 years ago

velihukka commented 4 years ago

Hi,

Would it be possible to choose to have the time at the top when choosing a frame be shown as mm.ss.ff as in Final Cut Pro X instead of mm.ss.cc (ff for frame, cc for centiseconds)? This would ideally be an option that could be toggled.

arthurhammer commented 4 years ago

Would require some changes in how video data is read but technically it's possible yeah. I haven't looked much into it but I think the AVAssetReader API is where we would get exact frame numbers from.

I don't have access to Final Cut Pro. Do you have a simple screenshot for me showing that time?

velihukka commented 4 years ago

Hi,

here's me scrubbing through a couple of seconds of 30 fps footage as a gif. ezgif-2-c516362a5b6b

arthurhammer commented 4 years ago

Thanks, got it! I'll definitely look into this in the future but I cannot promise any particular time frame right now.

arthurhammer commented 4 years ago

One more question, what's your use case for this, i.e. how does it help you knowing the exact frame number?

velihukka commented 4 years ago

It's honestly not much of a use case, it's just a way of representing the time in a way I (and others) are used to. I think its more logical i.e. when you have 24 fps video which means a frame every 0,0416 seconds which is a bit hard to show logically in fractions of a second

arthurhammer commented 4 years ago

Understood. I don't have much of a video editing background so this info is helpful. Thanks!

arthurhammer commented 4 years ago

I made some progress on this. The time label in Frame Grabber 1.2, just released, now displays exact frame times (commit 3afafe4).

Previously, it simply displayed the video player's current playback time (which can be anywhere in-between two distinct frame times).

From here, it shouldn't be too far to the "mm.ss.ff" time format. However, I still need to look into timecode and drop-frame semantics to understand how this works.

arthurhammer commented 3 years ago

I'm currently revisiting this topic. Here is what I understand so far. Feel free to correct any misunderstandings!

Timecodes (mm:ss.ff) are mostly used for videos with constant frames. A timecode for a frame can be calculated by the frame number and the frame rate. And there is a direct mapping from a timecode to the absolute time of a frame (presentation timestamp, mm:ss:SSS). (There are some nuances for non-integer frame rates like 29.97 where the drop-frame technique can be used.)

In videos with variable frame rates, it seems to work differently. Here, each frame is assigned its presentation timestamp (mm:ss:SSS) directly without relying on the frame rate. In contrast to constant frame rates, this requires indexing all frames in the video to find their presentation timestamps.

The question is how do we show timecodes (mm:ss.ff) for variable frame rates? My basic idea would be to simply group frame presentation time stamps by the full second they start in and then count each frame relative to that second.

Example: 

Presentation   "Timecode"
Timestamp

00:00:000      00:00.0 
00:00:300      00:00.1 
00:00:600      00:00.2
00:00:900      00:00.3 
00:01:200      00:01.0
00:01:500      00:01.1
00:01:800      00:01.2
00:02:100      00:02.0
...            ...

There are several issues with this. First, we lose the mapping from timecodes back to absolute times as we have in constant frame rate timecodes. Without further information, we don't know how far apart frames 00:00.1 and 00:00.2 are.

Second, timecodes are a standardized concept for constant frame rates. I'm not sure if we should use the same format (mm:ss.ff) with our custom adaption for variable frame rates. It might lead to more confusion for users familiar with the standard timecodes. We could show it in a custom format like mm:ss/ff to avoid confusion.

In any case, maybe there is already a convention for displaying timecodes for variable frame rate videos I haven't found yet. If there is, please let me know!

arthurhammer commented 3 years ago

This is now released in 1.3 with the caveats mentioned above.

These are not real timecodes according to the SMPTE standard but they are an adaption for variable frame rate videos. The frame time code is simply the frame number relative to the current second. Or: the index of the frame in all frames starting within the current second.

For constant frame rate videos with full frame rates (24, 30, 60 fps), this should actually be the exact same. For fractional frame rates like 29.97 fps, it usually isn't.