search

juanmcasillas / gopro2gpx

Parse the gpmd stream for GOPRO moov track (MP4) and extract the GPS info into a GPX (and kml) file.
GNU General Public License v3.0
197 stars 50 forks source link

Use JSON output from ffprobe #18

Closed alycda closed 2 years ago

alycda commented 3 years ago

use JSON-output of ffprobe for parsing instead of Regex to parse to Human-readable ouput

(I'm new to python, go easy on me)

still getting the following error:

% gopro2gpx ../_samples/test.mp4 test    
Traceback (most recent call last):
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/bin/gopro2gpx", line 33, in <module>
    sys.exit(load_entry_point('gopro2gpx==0.1', 'console_scripts', 'gopro2gpx')())
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/bin/gopro2gpx", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/__main__.py", line 3, in <module>
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/gopro2gpx.py", line 148, in main
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/gpmf.py", line 60, in readFromMP4
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/gpmf.py", line 99, in parseStream
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/klvdata.py", line 34, in __init__
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/fourCC.py", line 391, in Manage
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/fourCC.py", line 133, in Build
TypeError: a bytes-like object is required, not 'NoneType'

different file from same camera:

% gopro2gpx ../_samples/GH010201.mp4 GH010201     
GPSFIX change to 3 [lock 3D (ok)]
-- stats -----------------
- Ok:               6196
- GPSFIX=0 (bad):      0 (skipped: 0)
- Empty (No data):     0
Total points:       6196
--------------------------
Traceback (most recent call last):
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/bin/gopro2gpx", line 33, in <module>
    sys.exit(load_entry_point('gopro2gpx==0.1', 'console_scripts', 'gopro2gpx')())
TypeError: 'module' object is not callable

file split with ffmpeg:

gopro2gpx ../_samples/GH010201000.mp4 GH010201000
Traceback (most recent call last):
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/bin/gopro2gpx", line 33, in <module>
    sys.exit(load_entry_point('gopro2gpx==0.1', 'console_scripts', 'gopro2gpx')())
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/bin/gopro2gpx", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/__main__.py", line 3, in <module>
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/gopro2gpx.py", line 148, in main
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/gpmf.py", line 44, in readFromMP4
TypeError: cannot unpack non-iterable NoneType object
Simske commented 3 years ago

Thanks for the PR! I just tested it (on just a single file), and everything seems to work!

All the errors you are getting are related to the badly defined entrypoint, which is fixed in #15.

alycda commented 3 years ago

All the errors you are getting are related to badly the badly defined entrypoint, which is fixed in #15.

I copied over your changes from #15 into my local, I see the following resolved:

Traceback (most recent call last):
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/bin/gopro2gpx", line 33, in <module>
    sys.exit(load_entry_point('gopro2gpx==0.1', 'console_scripts', 'gopro2gpx')())
TypeError: 'module' object is not callable

But I guess my "test.mp4" is still corrupt:

% gopro2gpx ../_samples/test.mp4 test        
Traceback (most recent call last):
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/bin/gopro2gpx", line 33, in <module>
    sys.exit(load_entry_point('gopro2gpx==0.1', 'console_scripts', 'gopro2gpx')())
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/bin/gopro2gpx", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/__main__.py", line 3, in <module>
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/gopro2gpx.py", line 148, in main
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/gpmf.py", line 60, in readFromMP4
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/gpmf.py", line 103, in parseStream
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/klvdata.py", line 34, in __init__
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/fourCC.py", line 391, in Manage
  File "/Users/davisa/code/python/gopro2gpx/gopro2gpx-alyssa/lib/python3.9/site-packages/gopro2gpx-0.1-py3.9.egg/gopro2gpx/fourCC.py", line 133, in Build
TypeError: a bytes-like object is required, not 'NoneType'

I'll continue to dig to provide better error handling in this case. Thanks for your input!

Simske commented 3 years ago

Hm, curious. Have you tried extracting the data with another tool (https://goprotelemetryextractor.com/free/)?

Seems like your problem was entirely unrelated to ffprobe, there is something going wrong while decoding the data. Unfortunately I haven't looked into the details of decoding.

Can you upload the extracted data stream (as described in the Readme)?

alycda commented 3 years ago

Hm, curious. Have you tried extracting the data with another tool (https://goprotelemetryextractor.com/free/)?

Yes, I typically use this tool, however I was experimenting with my own custom lambda. I don't recall how I created "test.mp4" (pretty sure I just exported a 30 second clip from the GoPro iPhone app) so that I could work with a smaller file. The giant (4gb) file is working though, so I'll keep better track of how I source my files.

The source code for goprotelemetryextractor is available on github, https://github.com/JuanIrache/gopro-telemetry#readme, but NodeJS was running out of memory for large files. I believe test.mp4 worked with that tool, but I intended to write my lambda in python just as a learning experience (and it's not encountering memory errors).

JimmyS83 commented 2 years ago

@juanmcasillas

I have the same issue, 3.7GB, Hero 8, original file from camera https://goprotelemetryextractor.com/free/ exports GPX just fine... But gopro2gpx cant get gpx from it.

Traceback (most recent call last):
  File "D:\Program Files\Python3\Scripts\gopro2gpx-script.py", line 33, in <module>
    sys.exit(load_entry_point('gopro2gpx==0.1', 'console_scripts', 'gopro2gpx')())
  File "D:\Program Files\Python3\Scripts\gopro2gpx-script.py", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "D:\Program Files\Python3\lib\importlib\metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "D:\Program Files\Python3\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "D:\Program Files\Python3\lib\site-packages\gopro2gpx-0.1-py3.8.egg\gopro2gpx\__main__.py", line 3, in <module>
  File "D:\Program Files\Python3\lib\site-packages\gopro2gpx-0.1-py3.8.egg\gopro2gpx\gopro2gpx.py", line 148, in main
  File "D:\Program Files\Python3\lib\site-packages\gopro2gpx-0.1-py3.8.egg\gopro2gpx\gpmf.py", line 43, in readFromMP4
TypeError: cannot unpack non-iterable NoneType object

18 fixes this.