search

mapillary / mapillary_tools

Command line tools for processing and uploading Mapillary imagery
BSD 2-Clause "Simplified" License
259 stars 137 forks source link

Version 0.10.0 git test fail: test_process_gopro_hero8 test_video_process_sample_with_distance #597

Closed stefanbj closed 1 year ago

stefanbj commented 1 year ago

Basic information

Steps to reproduce behavior

  1. clone git repo -- (git clone git@github.com:mapillary/mapillary_tools.git etc...)
  2. git checkout f8bfbda06cf5018da4a2cacdec4e957faf64148a
  3. python3 -m pytest -s -vv tests

Reverting to tag v0.10.0 doesn't seem to make any difference.

Expected behavior

...

Actual behavior

=============================================== short test summary info ===============================================
FAILED tests/integration/test_gopro.py::test_process_gopro_hero8 - AssertionError: assert False
FAILED tests/integration/test_process.py::test_video_process_sample_with_distance - AssertionError: assert False
====================================== 2 failed, 134 passed in 152.70s (0:02:32) ======================================
P

1. test_process_gopro_hero8

PS C:\Users\stefanb\OUTDIR\mt\mapillary_tools> python3 -m pytest -s -vv tests/integration/test_gopro.py::test_process_gopro_hero8
================================================= test session starts =================================================
platform win32 -- Python 3.10.10, pytest-7.1.2, pluggy-1.0.0 -- C:\Users\stefanb\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\stefanb\OUTDIR\mt\mapillary_tools, configfile: setup.cfg
collecting ... ffmpeg version 5.1-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-shared --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil      57. 28.100 / 57. 28.100
libavcodec     59. 37.100 / 59. 37.100
libavformat    59. 27.100 / 59. 27.100
libavdevice    59.  7.100 / 59.  7.100
libavfilter     8. 44.100 /  8. 44.100
libswscale      6.  7.100 /  6.  7.100
libswresample   4.  7.100 /  4.  7.100
libpostproc    56.  6.100 / 56.  6.100
ffprobe version 5.1-full_build-www.gyan.dev Copyright (c) 2007-2022 the FFmpeg developers
built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-shared --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil      57. 28.100 / 57. 28.100
libavcodec     59. 37.100 / 59. 37.100
libavformat    59. 27.100 / 59. 27.100
libavdevice    59.  7.100 / 59.  7.100
libavfilter     8. 44.100 /  8. 44.100
libswscale      6.  7.100 /  6.  7.100
libswresample   4.  7.100 /  4.  7.100
libpostproc    56.  6.100 / 56.  6.100
collected 1 item

tests/integration/test_gopro.py::test_process_gopro_hero8 2023-02-21 22:13:48,313 - INFO    - Extracting video information: ffprobe -print_format json -hide_banner -show_format -show_streams C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-3\test_process_gopro_hero80\data\hero8.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-3\test_process_gopro_hero80\data\hero8.mp4':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    creation_time   : 2019-11-18T15:41:25.000000Z
    firmware        : HD8.01.01.20.00
  Duration: 00:00:12.65, start: 0.000000, bitrate: 2784 kb/s
  Stream #0:0[0x1](eng): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt709, progressive), 848x480 [SAR 1:1 DAR 53:30], 2503 kb/s, 29.97 fps, 29.97 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro AVC
      vendor_id       : [0][0][0][0]
      encoder         : GoPro AVC encoder
      timecode        : 15:40:28:28
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 188 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro AAC
      vendor_id       : [0][0][0][0]
      timecode        : 15:40:28:28
  Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro TCD
      timecode        : 15:40:28:28
  Stream #0:3[0x4](eng): Data: bin_data (gpmd / 0x646D7067), 54 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro MET
  Stream #0:4[0x5](eng): Data: none (fdsc / 0x63736466), 10 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro SOS
Unsupported codec with id 0 for input stream 2
Unsupported codec with id 98314 for input stream 3
Unsupported codec with id 0 for input stream 4
2023-02-21 22:13:48,380 - INFO    - Extracting frames: ffmpeg -hide_banner -nostdin -i C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-3\test_process_gopro_hero80\data\hero8.mp4 -vf fps=1/2.0 -qscale:v 2 C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-3\test_process_gopro_hero80\data\mapillary_sampled_video_frames\.mly_ffmpeg_hero8.mp4_2412_1677014028\hero8_NA_%06d.jpg
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-3\test_process_gopro_hero80\data\hero8.mp4':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    creation_time   : 2019-11-18T15:41:25.000000Z
    firmware        : HD8.01.01.20.00
  Duration: 00:00:12.65, start: 0.000000, bitrate: 2784 kb/s
  Stream #0:0[0x1](eng): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt709, progressive), 848x480 [SAR 1:1 DAR 53:30], 2503 kb/s, 29.97 fps, 29.97 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro AVC
      vendor_id       : [0][0][0][0]
      encoder         : GoPro AVC encoder
      timecode        : 15:40:28:28
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 188 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro AAC
      vendor_id       : [0][0][0][0]
      timecode        : 15:40:28:28
  Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro TCD
      timecode        : 15:40:28:28
  Stream #0:3[0x4](eng): Data: bin_data (gpmd / 0x646D7067), 54 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro MET
  Stream #0:4[0x5](eng): Data: none (fdsc / 0x63736466), 10 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro SOS
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Output #0, image2, to 'C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-3\test_process_gopro_hero80\data\mapillary_sampled_video_frames\.mly_ffmpeg_hero8.mp4_2412_1677014028\hero8_NA_%06d.jpg':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    firmware        : HD8.01.01.20.00
    encoder         : Lavf59.27.100
  Stream #0:0(eng): Video: mjpeg, yuvj420p(pc, bt709, progressive), 848x480 [SAR 1:1 DAR 53:30], q=2-31, 200 kb/s, 0.50 fps, 0.50 tbn (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro AVC
      vendor_id       : [0][0][0][0]
      timecode        : 15:40:28:28
      encoder         : Lavc59.37.100 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
frame=    6 fps=0.0 q=2.0 Lsize=N/A time=00:00:12.00 bitrate=N/A speed=  43x
video:552kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
2023-02-21 22:13:48,737 - WARNING - Force the option "filetypes" to be "image" to avoid processing and uploading both the video samples and the videos themselves
Interpolating hero8.mp4: 100%|██████████████████████████████████████████████████████| 6/6 [00:00<00:00, 214.41images/s]
Verifying image EXIF writing: 100%|█████████████████████████████████████████████████| 6/6 [00:00<00:00, 859.78images/s]
2023-02-21 22:13:49,069 - INFO    - Check the description file for details: C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-3\test_process_gopro_hero80\data\mapillary_sampled_video_frames\mapillary_image_description.json
2023-02-21 22:13:49,069 - INFO    -        6 image(s) read in total
2023-02-21 22:13:49,069 - INFO    -             6 image(s) are ready to be uploaded
FAILED

====================================================== FAILURES =======================================================
______________________________________________ test_process_gopro_hero8 _______________________________________________

setup_data = local('C:\\Users\\stefanb\\AppData\\Local\\Temp\\pytest-of-stefanb\\pytest-3\\test_process_gopro_hero80\\data')

    @pytest.mark.usefixtures("setup_config")
    @pytest.mark.usefixtures("setup_upload")
    @pytest.mark.usefixtures("setup_envvars")
    def test_process_gopro_hero8(
        setup_data: py.path.local,
    ):
        if not is_ffmpeg_installed:
            pytest.skip("skip because ffmpeg not installed")
        video_path = setup_data.join("hero8.mp4")
        # this sample hero8.mp4 doesn't have any good GPS points,
        # so we do not filter out bad GPS points
        x = subprocess.run(
            f"{EXECUTABLE} video_process --video_sample_interval=2 --video_sample_distance=-1 --geotag_source=gopro_videos {str(video_path)}",
            shell=True,
        )
        assert x.returncode == 0, x.stderr
        desc_path = setup_data.join("mapillary_sampled_video_frames").join(
            "mapillary_image_description.json"
        )
        assert desc_path.exists()
        with open(desc_path) as fp:
            descs = json.load(fp)
        for expected, actual in zip(expected_descs, descs):
            assert abs(expected["MAPLatitude"] - actual["MAPLatitude"]) < 0.0000001
            assert abs(expected["MAPLongitude"] - actual["MAPLongitude"]) < 0.0000001
            assert expected["MAPCaptureTime"] == actual["MAPCaptureTime"]
            assert abs(expected["MAPAltitude"] - actual["MAPAltitude"]) < 0.001
            assert (
                abs(
                    expected["MAPCompassHeading"]["TrueHeading"]
                    - actual["MAPCompassHeading"]["TrueHeading"]
                )
                < 0.001
            )
            assert (
                abs(
                    expected["MAPCompassHeading"]["MagneticHeading"]
                    - actual["MAPCompassHeading"]["MagneticHeading"]
                )
                < 0.001
            )
            assert Path(actual["filename"]).is_file(), actual["filename"]
>           assert actual["filename"].endswith(expected["filename"])
E           AssertionError: assert False
E            +  where False = <built-in method endswith of str object at 0x000002917E4F3130>('hero8.mp4/hero8_NA_000001.jpg')
E            +    where <built-in method endswith of str object at 0x000002917E4F3130> = 'C:\\Users\\stefanb\\AppData\\Local\\Temp\\pytest-of-stefanb\\pytest-3\\test_process_gopro_hero80\\data\\mapillary_sampled_video_frames\\hero8.mp4\\hero8_NA_000001.jpg'.endswith

tests\integration\test_gopro.py:148: AssertionError
=============================================== short test summary info ===============================================
FAILED tests/integration/test_gopro.py::test_process_gopro_hero8 - AssertionError: assert False
================================================== 1 failed in 2.32s ==================================================
PS C:\Users\stefanb\OUTDIR\mt\mapillary_tools>

2. test_video_process_sample_with_distance

PS C:\Users\stefanb\OUTDIR\mt\mapillary_tools> python3 -m pytest -s -vv tests/integration/test_process.py::test_video_process_sample_with_distance
================================================= test session starts =================================================
platform win32 -- Python 3.10.10, pytest-7.1.2, pluggy-1.0.0 -- C:\Users\stefanb\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\stefanb\OUTDIR\mt\mapillary_tools, configfile: setup.cfg
collecting ... ffmpeg version 5.1-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-shared --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil      57. 28.100 / 57. 28.100
libavcodec     59. 37.100 / 59. 37.100
libavformat    59. 27.100 / 59. 27.100
libavdevice    59.  7.100 / 59.  7.100
libavfilter     8. 44.100 /  8. 44.100
libswscale      6.  7.100 /  6.  7.100
libswresample   4.  7.100 /  4.  7.100
libpostproc    56.  6.100 / 56.  6.100
ffprobe version 5.1-full_build-www.gyan.dev Copyright (c) 2007-2022 the FFmpeg developers
built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-shared --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil      57. 28.100 / 57. 28.100
libavcodec     59. 37.100 / 59. 37.100
libavformat    59. 27.100 / 59. 27.100
libavdevice    59.  7.100 / 59.  7.100
libavfilter     8. 44.100 /  8. 44.100
libswscale      6.  7.100 /  6.  7.100
libswresample   4.  7.100 /  4.  7.100
libpostproc    56.  6.100 / 56.  6.100
collected 1 item

tests/integration/test_process.py::test_video_process_sample_with_distance 2023-02-21 22:15:39,973 - DEBUG   - mapillary_tools version 0.10.0.git
2023-02-21 22:15:39,973 - DEBUG   - CLI param: verbose: True
2023-02-21 22:15:39,974 - DEBUG   - CLI param: video_import_path: C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-4\test_video_process_sample_with0\data\gopro_data
2023-02-21 22:15:39,974 - DEBUG   - CLI param: import_path: C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-4\test_video_process_sample_with0\data\gopro_data\my_samples
2023-02-21 22:15:39,974 - DEBUG   - CLI param: video_sample_distance: 6.0
2023-02-21 22:15:39,974 - DEBUG   - CLI param: video_sample_interval: -1.0
2023-02-21 22:15:39,974 - DEBUG   - CLI param: video_duration_ratio: 1.0
2023-02-21 22:15:39,975 - DEBUG   - CLI param: skip_subfolders: False
2023-02-21 22:15:39,975 - DEBUG   - CLI param: rerun: False
2023-02-21 22:15:39,975 - DEBUG   - CLI param: skip_sample_errors: False
2023-02-21 22:15:39,975 - DEBUG   - CLI param: skip_process_errors: False
2023-02-21 22:15:39,975 - DEBUG   - CLI param: filetypes: camm, gopro, image, blackvue
2023-02-21 22:15:39,975 - DEBUG   - CLI param: overwrite_all_EXIF_tags: False
2023-02-21 22:15:39,975 - DEBUG   - CLI param: overwrite_EXIF_time_tag: False
2023-02-21 22:15:39,975 - DEBUG   - CLI param: overwrite_EXIF_gps_tag: False
2023-02-21 22:15:39,975 - DEBUG   - CLI param: overwrite_EXIF_direction_tag: False
2023-02-21 22:15:39,975 - DEBUG   - CLI param: overwrite_EXIF_orientation_tag: False
2023-02-21 22:15:39,976 - DEBUG   - CLI param: add_file_name: False
2023-02-21 22:15:39,976 - DEBUG   - CLI param: add_import_date: True
2023-02-21 22:15:39,976 - DEBUG   - CLI param: geotag_source: exif
2023-02-21 22:15:39,976 - DEBUG   - CLI param: interpolation_use_gpx_start_time: False
2023-02-21 22:15:39,976 - DEBUG   - CLI param: interpolation_offset_time: 0.0
2023-02-21 22:15:39,976 - DEBUG   - CLI param: offset_angle: 0.0
2023-02-21 22:15:39,976 - DEBUG   - CLI param: offset_time: 0.0
2023-02-21 22:15:39,976 - DEBUG   - CLI param: cutoff_distance: 600.0
2023-02-21 22:15:39,976 - DEBUG   - CLI param: cutoff_time: 60.0
2023-02-21 22:15:39,976 - DEBUG   - CLI param: interpolate_directions: False
2023-02-21 22:15:39,976 - DEBUG   - CLI param: duplicate_distance: 0.1
2023-02-21 22:15:39,977 - DEBUG   - CLI param: duplicate_angle: 5.0
2023-02-21 22:15:39,979 - DEBUG   - Found 2 videos in C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-4\test_video_process_sample_with0\data\gopro_data
2023-02-21 22:15:39,980 - INFO    - Extracting video information: ffprobe -print_format json -hide_banner -show_format -show_streams C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-4\test_video_process_sample_with0\data\gopro_data\hero8.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-4\test_video_process_sample_with0\data\gopro_data\hero8.mp4':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    creation_time   : 2019-11-18T15:41:25.000000Z
    firmware        : HD8.01.01.20.00
  Duration: 00:00:12.65, start: 0.000000, bitrate: 2784 kb/s
  Stream #0:0[0x1](eng): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt709, progressive), 848x480 [SAR 1:1 DAR 53:30], 2503 kb/s, 29.97 fps, 29.97 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro AVC
      vendor_id       : [0][0][0][0]
      encoder         : GoPro AVC encoder
      timecode        : 15:40:28:28
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 188 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro AAC
      vendor_id       : [0][0][0][0]
      timecode        : 15:40:28:28
  Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro TCD
      timecode        : 15:40:28:28
  Stream #0:3[0x4](eng): Data: bin_data (gpmd / 0x646D7067), 54 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro MET
  Stream #0:4[0x5](eng): Data: none (fdsc / 0x63736466), 10 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:41:25.000000Z
      handler_name    : GoPro SOS
Unsupported codec with id 0 for input stream 2
Unsupported codec with id 98314 for input stream 3
Unsupported codec with id 0 for input stream 4
2023-02-21 22:15:40,040 - DEBUG   - Extracted video duration: 12.645967
2023-02-21 22:15:40,040 - DEBUG   - Extracted video creation time: 2019-11-18T15:41:25.000000Z
2023-02-21 22:15:40,043 - INFO    - Extracting video metdata
2023-02-21 22:15:40,299 - DEBUG   - Removed 231 points with the GPS fix not in {2, 3}
2023-02-21 22:15:40,300 - WARNING - Empty GPS data found
2023-02-21 22:15:40,301 - INFO    - Extracting video information: ffprobe -print_format json -hide_banner -show_format -show_streams C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-4\test_video_process_sample_with0\data\gopro_data\max-360mode.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-4\test_video_process_sample_with0\data\gopro_data\max-360mode.mp4':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    creation_time   : 2019-11-18T15:45:00.000000Z
    location        : +33.1267-117.3273/
    location-eng    : +33.1267-117.3273/
    firmware        : H19.03.01.30.00
  Duration: 00:00:12.14, start: 0.000000, bitrate: 2816 kb/s
  Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709, progressive), 1408x704 [SAR 1:1 DAR 2:1], 2519 kb/s, 23.98 fps, 23.98 tbr, 24k tbn (default)
    Metadata:
      creation_time   : 2019-11-18T15:45:00.000000Z
      handler_name    : GoPro AVC
      vendor_id       : [0][0][0][0]
      encoder         : GoPro AVC encoder
      timecode        : 15:44:03:21
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 188 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:45:00.000000Z
      handler_name    : GoPro AAC
      vendor_id       : [0][0][0][0]
      timecode        : 15:44:03:21
  Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:45:00.000000Z
      handler_name    : GoPro TCD
      timecode        : 15:44:03:21
  Stream #0:3[0x4](eng): Data: bin_data (gpmd / 0x646D7067), 81 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:45:00.000000Z
      handler_name    : GoPro MET
  Stream #0:4[0x5](eng): Data: none (fdsc / 0x63736466), 10 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:45:00.000000Z
      handler_name    : GoPro SOS
Unsupported codec with id 0 for input stream 2
Unsupported codec with id 98314 for input stream 3
Unsupported codec with id 0 for input stream 4
2023-02-21 22:15:40,365 - DEBUG   - Extracted video duration: 12.137125
2023-02-21 22:15:40,367 - DEBUG   - Extracted video creation time: 2019-11-18T15:45:00.000000Z
2023-02-21 22:15:40,367 - INFO    - Extracting video metdata
2023-02-21 22:15:40,656 - DEBUG   - max distance: 0.094214
2023-02-21 22:15:40,657 - DEBUG   - Split to 1 sequences with max distance 30.000000
2023-02-21 22:15:40,658 - DEBUG   - Found 1 sequences after merging with max speed 1.641000
2023-02-21 22:15:40,659 - INFO    - Found total 211 GPS points
2023-02-21 22:15:40,659 - INFO    - Extracting video samples
2023-02-21 22:15:40,662 - INFO    - Extracting video samples
2023-02-21 22:15:40,664 - INFO    - Found total 291 video samples
2023-02-21 22:15:40,665 - INFO    - Interpolating video samples in the time range from 0.0 to 11.622416666666666
2023-02-21 22:15:40,666 - INFO    - Found total 279 interpolated video samples
2023-02-21 22:15:40,668 - INFO    - Selected 3 video samples by the minimal sample distance 6.0
2023-02-21 22:15:40,670 - INFO    - Extracting frames: ffmpeg -hide_banner -nostdin -i C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-4\test_video_process_sample_with0\data\gopro_data\max-360mode.mp4 -map 0:0 -vf select=eq(n\,0)+eq(n\,127)+eq(n\,250) -vsync 0 -frames:0 3 -frame_pts 1 -qscale:0 2 C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-4\test_video_process_sample_with0\data\gopro_data\my_samples\.mly_ffmpeg_max-360mode.mp4_4160_1677014140\max-360mode_0_%06d.jpg
-vsync is deprecated. Use -fps_mode
Passing a number to -vsync is deprecated, use a string argument as described in the manual.
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-4\test_video_process_sample_with0\data\gopro_data\max-360mode.mp4':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    creation_time   : 2019-11-18T15:45:00.000000Z
    location        : +33.1267-117.3273/
    location-eng    : +33.1267-117.3273/
    firmware        : H19.03.01.30.00
  Duration: 00:00:12.14, start: 0.000000, bitrate: 2816 kb/s
  Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709, progressive), 1408x704 [SAR 1:1 DAR 2:1], 2519 kb/s, 23.98 fps, 23.98 tbr, 24k tbn (default)
    Metadata:
      creation_time   : 2019-11-18T15:45:00.000000Z
      handler_name    : GoPro AVC
      vendor_id       : [0][0][0][0]
      encoder         : GoPro AVC encoder
      timecode        : 15:44:03:21
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 188 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:45:00.000000Z
      handler_name    : GoPro AAC
      vendor_id       : [0][0][0][0]
      timecode        : 15:44:03:21
  Stream #0:2[0x3](eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:45:00.000000Z
      handler_name    : GoPro TCD
      timecode        : 15:44:03:21
  Stream #0:3[0x4](eng): Data: bin_data (gpmd / 0x646D7067), 81 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:45:00.000000Z
      handler_name    : GoPro MET
  Stream #0:4[0x5](eng): Data: none (fdsc / 0x63736466), 10 kb/s (default)
    Metadata:
      creation_time   : 2019-11-18T15:45:00.000000Z
      handler_name    : GoPro SOS
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Output #0, image2, to 'C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-4\test_video_process_sample_with0\data\gopro_data\my_samples\.mly_ffmpeg_max-360mode.mp4_4160_1677014140\max-360mode_0_%06d.jpg':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    firmware        : H19.03.01.30.00
    location        : +33.1267-117.3273/
    location-eng    : +33.1267-117.3273/
    encoder         : Lavf59.27.100
  Stream #0:0(eng): Video: mjpeg, yuvj420p(pc, bt709, progressive), 1408x704 [SAR 1:1 DAR 2:1], q=2-31, 200 kb/s, 23.98 fps, 23.98 tbn (default)
    Metadata:
      creation_time   : 2019-11-18T15:45:00.000000Z
      handler_name    : GoPro AVC
      vendor_id       : [0][0][0][0]
      timecode        : 15:44:03:21
      encoder         : Lavc59.37.100 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
frame=    3 fps=0.0 q=2.0 Lsize=N/A time=00:00:10.46 bitrate=N/A speed=36.9x
video:396kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
2023-02-21 22:15:41,035 - WARNING - Force the option "filetypes" to be "image" to avoid processing and uploading both the video samples and the videos themselves
2023-02-21 22:15:41,061 - WARNING - The option --add_import_date is not needed any more since v0.10.0
2023-02-21 22:15:41,088 - INFO    - Check the description file for details: C:\Users\stefanb\AppData\Local\Temp\pytest-of-stefanb\pytest-4\test_video_process_sample_with0\data\gopro_data\my_samples\mapillary_image_description.json
2023-02-21 22:15:41,088 - INFO    -        3 image(s) read in total
2023-02-21 22:15:41,089 - INFO    -             3 image(s) are ready to be uploaded
FAILED

====================================================== FAILURES =======================================================
_______________________________________ test_video_process_sample_with_distance _______________________________________

setup_data = local('C:\\Users\\stefanb\\AppData\\Local\\Temp\\pytest-of-stefanb\\pytest-4\\test_video_process_sample_with0\\data')

    def test_video_process_sample_with_distance(setup_data: py.path.local):
        if not is_ffmpeg_installed:
            pytest.skip("skip because ffmpeg not installed")

        video_dir = setup_data.join("gopro_data")
        desc_path = video_dir.join("my_samples").join("mapillary_image_description.json")
        for option in [
            "--video_sample_distance=6",
            "--video_sample_distance=6 --video_sample_interval=-2",
        ]:
            x = subprocess.run(
                f"{EXECUTABLE} --verbose video_process {option} {PROCESS_FLAGS} {video_dir} {video_dir.join('my_samples')}",
                shell=True,
            )
            assert x.returncode == 0, x.stderr
            expected = [
                {
                    "filename": "gopro_data/my_samples/max-360mode.mp4/max-360mode_0_000000.jpg",
                    "filetype": "image",
                    "MAPLatitude": 33.1266719,
                    "MAPLongitude": -117.3273063,
                    "MAPCaptureTime": "2019_11_18_15_44_47_862",
                    "MAPAltitude": -22.18,
                    "MAPCompassHeading": {"TrueHeading": 313.68, "MagneticHeading": 313.68},
                    "MAPSequenceUUID": "0",
                    "MAPDeviceMake": "GoPro",
                    "MAPDeviceModel": "GoPro Max",
                    "MAPOrientation": 1,
                },
                {
                    "filename": "gopro_data/my_samples/max-360mode.mp4/max-360mode_0_000127.jpg",
                    "filetype": "image",
                    "MAPLatitude": 33.1267206,
                    "MAPLongitude": -117.3273345,
                    "MAPCaptureTime": "2019_11_18_15_44_53_159",
                    "MAPAltitude": -21.91,
                    "MAPCompassHeading": {"TrueHeading": 330.82, "MagneticHeading": 330.82},
                    "MAPSequenceUUID": "0",
                    "MAPDeviceMake": "GoPro",
                    "MAPDeviceModel": "GoPro Max",
                    "MAPOrientation": 1,
                },
                {
                    "filename": "gopro_data/my_samples/max-360mode.mp4/max-360mode_0_000250.jpg",
                    "filetype": "image",
                    "MAPLatitude": 33.1267702,
                    "MAPLongitude": -117.3273612,
                    "MAPCaptureTime": "2019_11_18_15_44_58_289",
                    "MAPAltitude": -22.58,
                    "MAPCompassHeading": {"TrueHeading": 10.54, "MagneticHeading": 10.54},
                    "MAPSequenceUUID": "0",
                    "MAPDeviceMake": "GoPro",
                    "MAPDeviceModel": "GoPro Max",
                    "MAPOrientation": 1,
                },
            ]
            with open(desc_path) as fp:
                descs = json.load(fp)
            for e, d in zip(expected, descs):
>               assert d["filename"].endswith(e["filename"])
E               AssertionError: assert False
E                +  where False = <built-in method endswith of str object at 0x000001A542FEFE50>('gopro_data/my_samples/max-360mode.mp4/max-360mode_0_000000.jpg')
E                +    where <built-in method endswith of str object at 0x000001A542FEFE50> = 'C:\\Users\\stefanb\\AppData\\Local\\Temp\\pytest-of-stefanb\\pytest-4\\test_video_process_sample_with0\\data\\gopro_data\\my_samples\\max-360mode.mp4\\max-360mode_0_000000.jpg'.endswith

tests\integration\test_process.py:582: AssertionError
=============================================== short test summary info ===============================================
FAILED tests/integration/test_process.py::test_video_process_sample_with_distance - AssertionError: assert False
================================================== 1 failed in 2.18s ==================================================
PS C:\Users\stefanb\OUTDIR\mt\mapillary_tools>

Corresponding data

N/A

Additional information

N/A

stefanbj commented 1 year ago

I believe the tests/integration/test_process.py::test_video_process_sample_with_distance test failure is causing the deprecated option --video_sample_interval 3.0 effectively be unusable, as attempting to use it this results in an error: 2023-02-21 21:53:27,170 - ERROR - MapillaryBadParameterError: Expect either non-negative video_sample_distance or non-negative video_sample_interval but got 3.0 and 3.0 respectively

ptpt commented 1 year ago

It's because some assertion errors on Windows path... Fixed in the PR above. It wasn't reported by CI because GitHub Actions doesn't have ffmpeg installed so the test is not run.

Thanks for reporting again.

stefanbj commented 1 year ago

Confirmed, tests now also works for me.