Calculating each frame's timestamp

[fushi219]

Hi, I have a problem handing gopro max's 360° mp4 video. Using the gpmf-parser handing the mp4 file, I got the following output in the begining:

59.940 with 659 frames

And at last, i got: (i have changed 'SHUT' to 'CORI', so: double rate = GetGPMFSampleRate(cbobject, fourcc, STR2FOURCC("CORI"), GPMF_SAMPLE_RATE_PRECISE, &start, &end);)

COMPUTED SAMPLERATES: ACCL sampling rate = 198.625122Hz (time 0.001112 to 11.006769)", GYRO sampling rate = 794.500193Hz (time 0.000278 to 11.007198)", MAGN sampling rate = 24.828140Hz (time 0.019217 to 11.014805)", SHUT sampling rate = 29.979163Hz (time 0.017413 to 10.991702)", WBAL sampling rate = 9.989650Hz (time 0.021445 to 11.032842)", WRGB sampling rate = 9.989650Hz (time 0.021445 to 11.032842)", ISOE sampling rate = 29.979677Hz (time 0.015801 to 11.023258)", UNIF sampling rate = 9.989650Hz (time 0.021445 to 11.032842)", GPS5 sampling rate = 18.183566Hz (time 0.023392 to 10.967340)", CORI sampling rate = 60.053363Hz (time 0.000000 to 11.006877)", IORI sampling rate = 60.053363Hz (time 0.000000 to 11.006877)", DISP sampling rate = 60.053363Hz (time 0.000000 to 11.006877)", GRAV sampling rate = 60.053363Hz (time 0.000000 to 11.006877)", WNDM sampling rate = 10.000000Hz (time 0.000000 to 11.000000)", MWET sampling rate = 10.000000Hz (time 0.000000 to 11.000000)", AALP sampling rate = 10.000000Hz (time 0.000000 to 11.000000)",

As the CORI says, frame count is : (11.006877 - 0.00000) * 60.053363 = 660.999979977351, almost as 661 frame, however, the mp4 handler shows that it has only 659 frames.

So i wonder which method i should use? Thanks!

[dnewman-gpsw]

CORI is frame sync'd. So the first CORI value is for the first frame, second for the for the second, etc. Sometimes you get few extras. CORI is at the frame rate.

[fushi219]

Thanks a lot !!! So that's to say: I could use ffmpeg to extract each frame in mp4, then use incremental CORI timestamp as each image's accurate timestamp, is this right?

[dnewman-gpsw]

yes

[pjmara]

Is SHUT not frame sync'd? If you could comment also on how the few extras happen (beginning? end? random?) from either CORI or SHUT. Trying to accurately stamp each frame so want to know how I can connect SHUT/CORI messages with GPS5 messages. Regarding the STMPs- are SHUT/CORI messages stamped with STMP at start or end of exposure? And does the GPS5 STMP message match up with the GPSU time published in the GPS5 stream?

[dnewman-gpsw]

SHUT is not at the frame rate for all shooting mode for most camera models. If you are only using 24/25 or 30fps then SHUT is frame sync'd, otherwise CORI is the best. Start of the exposure. There is a bug in GPSU and GPS5 timestamping, it is occurs in the first couple of payloads, observable as the GPSU gap between the early payloads is wrong. For HERO10 and earlier, use a later payload and compute back. This was fixed in HERO11 which adds GPS9, which the time is presented in each sample, not just each payload.

[pjmara]

Thanks so much, after asking I realized there a bunch of similar threads (closed though) in github issues so appreciate your patience. Thanks for all of your good work on maintaining this repo. Much appreciated.

[pjmara]

Also, to clarify, does the CORI/SHUT message have a similar problem for microsecond stamp near the beginning of the recording? Or can that one be trusted?

[dnewman-gpsw]

They do not. The timing error is only for time varying information stored as "STICKY" class data. So the GPSU, GPSP and GPSF only.

[pjmara]

Sometimes you get few extras.

Are these extras at the beginning? At the end?

[dnewman-gpsw]

at the end.

[pjmara]

I know it's been a while, but the project I was working on just increased its timestamp accuracy needs, so I'm re-visiting this. I'm curious as to what the STMP field within the GPSU message means- does it correspond to the time that this gps message is received from the GPS? The reason I ask is I'm trying to establish a relationship between the GPS time in the GPSU message and the microseconds clock (STMP) for the video so that I can effectively give a frame (1 per CORI message) an accurate GPS timestamp. Is this even possible, or am I going down the wrong track?

[dnewman-gpsw]

This has since been solved on HERO11 cameras. We added GPS9, which includes the GPS time per sample, so the GPS time and STMPs now have a reliable relationship. Unfortunately older cameras with GPS5, will always have some timing error.

[pjmara]

Dang! I have a pile of historical data, some with hero8 and some hero10 that I'm going back in post and performing some additional analysis. Is the timing error completely undefined? I guess I am looking for best approximations on old data that does not benefit from the Hero11 improvements. If I can characterize the error that used to be occuring I could at least try and build a decent model for mitigating it. (Also I get that these may be specific questions for my project- so understand if you don't have time for a deep dive)