ElvinC / gyroflow

[INACTIVE LEGACY VERSION, FIND THE PROJECT HERE: https://github.com/gyroflow/gyroflow] Video stabilization using IMU motion data from internal or external logs
http://gyroflow.xyz
GNU General Public License v3.0
612 stars 89 forks source link

Error when using Hero8 or Hero9 footage over 4GB (labs firmware) #57

Open MiniGod opened 3 years ago

MiniGod commented 3 years ago

The GoPro Hero 8 and Hero 9 use H265 hevc codec when using "High" bitrate setting, and high resolution (2.7K 60, or 4K, etc).

Gyroflow doesn't seem to support it. When trying to use H265 footage from Hero8 or Hero9, I get the following error in the terminal when I press "Apply settings and compute sync":


Edit: h265 works. This error comes when using 12GB chapters enabled with Labs firmware.

Preset name: GoPro_Hero7__2.7K 4by3 wide
Note: 
Made with 13 frames using calibrator version 0.2.0-alpha on date 2021-02-06
Warning: Versions don't match. Calibrator: 0.3.0-beta/dev, Preset: 0.2.0-alpha
[warn] [/atom[2]/movie/atom[1]/udta/atom[0]/tag] Unable to convert string to Unicode: 'ascii' codec can't decode byte 0xa9 in position 0: ordinal not in range(128)
[warn] [/atom[2]/movie/atom[3]/track/atom[3]/media/atom[1]/hdlr] [Autofix] Delete '/atom[2]/movie/atom[3]/track/atom[3]/media/atom[1]/hdlr/name' (too large)
[warn] [/atom[2]/movie/atom[3]/track/atom[3]/media/atom[1]/hdlr] [Autofix] Fix parser error: stop parser, add padding
[warn] [/atom[2]/movie/atom[4]/track/atom[2]/media/atom[1]/hdlr] [Autofix] Delete '/atom[2]/movie/atom[4]/track/atom[2]/media/atom[1]/hdlr/name' (too large)
[warn] [/atom[2]/movie/atom[4]/track/atom[2]/media/atom[1]/hdlr] [Autofix] Fix parser error: stop parser, add padding
[warn] [/atom[2]/movie/atom[4]/track/atom[2]/media/atom[2]/minf/atom[2]/stbl/atom[0]/stsd/sample_entry[0]/atom[0]/esds/ES/decConfigDescr/reserved] padding contents doesn't look normal (invalid pattern)
[warn] [/atom[2]/movie/atom[5]/track/atom[1]/media/atom[1]/hdlr] [Autofix] Delete '/atom[2]/movie/atom[5]/track/atom[1]/media/atom[1]/hdlr/name' (too large)
[warn] [/atom[2]/movie/atom[5]/track/atom[1]/media/atom[1]/hdlr] [Autofix] Fix parser error: stop parser, add padding
<generator object get_payloads at 0x7f45a6719270>
Traceback (most recent call last):
  File "gyroflow.py", line 1966, in recompute_stab
    self.stab = stabilizer.GPMFStabilizer(self.infile_path, self.preset_path, gyro_path=self.gyro_log_path, hero=heronum, fov_scale=fov_val, gyro_lpf_cutoff = gyro_lpf)
  File "/home/kristjan/gyroflow/stabilizer.py", line 979, in __init__
    self.gpmf = Extractor(gyro_path)
  File "/home/kristjan/gyroflow/GPMF_gyro.py", line 20, in __init__
    for gpmf_data, timestamps in payloads:
  File "/home/kristjan/gyroflow/gpmf/extract.py", line 60, in get_payloads
    offset = stco["chunk_offset[{}]".format(idx)].value
UnboundLocalError: local variable 'stco' referenced before assignment
ElvinC commented 3 years ago

Huh, haven't seen that before. That's an error with the metadata extraction, so it seems that the GPMF parser library doesn't work with it (the video processing shouldn't be a problem since it's just ffmpeg). As a test, does it work fine with rsgo? Just to make sure the gyro data is fine.

kafie1980 commented 3 years ago

I have tested the GoPro Hero 8 Black using the preset I made for it and I did not come across these errors : https://github.com/ElvinC/gyroflow/blob/master/camera_presets/GoPro/GoPro_Hero_8_Black_2_7K_4by3_60fps_wide.json

I did test it at 2.7K 4x3 60fps wide fov. I am pretty sure it saved the .MP4 file in HEVC/H.265 codec. And it seemed to work perfectly.

I do not care much about 4k because its too much to process.

You seem to be using the GoPro_Hero7__2.7K 4by3 wide preset. Can you test with the Hero 8 preset and report back your results?

Not sure about the Gopro Hero 9 because I do not own one.

MiniGod commented 3 years ago

Sorry, it looks like h265 works. It might be related to GoPro labs firmware and 12GiB file limit?

With Labs firmware on the GoPro's, you can enable 12GiB file limit. With that enabled, files split at ~16min (12GiB) instead of ~5:20 (4GiB).

I seem to only get this error with files longer than ~5:20 (around 4GiB). I'll have to test more to know for sure that this is the cause.

@ElvinC : These files work in rsgo. @kafie1980 : I didn't know about that preset. It's missing on the dev branch. Using that preset didn't change anything.

ElvinC commented 3 years ago

Huh interesting. Let me know if you figure out more

kafie1980 commented 3 years ago

I have labs on my Gopro 8 but I have not enabled the 12gb file limit and my flights are usually under 5 minutes.

Unless you are sure the lens on the Gopro 7 and 8 are the same which I am not completely sure, I would suggest you use the present made for the Gopro 8. Else you might have distortions in your final results.

MiniGod commented 3 years ago

Oh yeah, sure. I only meant that changing the preset didn't solve the error.

kafie1980 commented 3 years ago

We are also not able to playback on the Gopro or usb transfer 12gb files and only sdcard copy is supported: https://gopro.github.io/labs/control/chapters/

MiniGod commented 3 years ago

This is definitely related to the mp4 files being over 4GB.

I tried encoding the "broken" file with lower bitrate (to get below 4GB), and it worked. And tried encoding a working file with higher bitrate (to get above 4GB), and I got the error.

So to reproduce this, you need to encode a gopro file with high enough bitrate to get above 4GB.

I was able to use the original h265 file as input file, and the below-4gb file for gyro. That way I can use the original file in gyroflow. I guess that works as a workaround for now.

To improve the workaround, I tried making a mp4 file with only the GPMF track mapped (only -map 0:3) to use as the gyro info, but then I got a different error in gyroflow.