mapillary / mapillary_tools

Command line tools for processing and uploading Mapillary imagery
BSD 2-Clause "Simplified" License
256 stars 134 forks source link

mapillary_tools failed to process Blackvue videos, unrecognized gps? #665

Open govvin opened 7 months ago

govvin commented 7 months ago

Basic information

Steps to reproduce behavior

  1. run mapillary_tools video_process_and_upload for a video file. (see console messages, last section of this ticket)
  2. process stops, and report that "empty GPS found"
  3. after the error, I tried the Desktop Uploader and the file was uploaded without issues

Expected behavior

CLI should work if Desktop Uploader (DU) can upload the videos without errors.

After the CLI error, I tried the Desktop Uploader and the same file(s) were successfully uploaded using the DU 4.04 image

I also tried other files with DU Beta 4.1.8, and it also worked without errors.

Actual behavior

Command line processing and upload doesn't work. The upload was successful using the DU, and it appeared in my upload list as being "ingested", after a day the sequence disappeared from my upload list.

I thought I'd try again, but this time DU refused to upload the files, reporting that they've already been uploaded: image. However, I can't find it from my uploads.

Corresponding data

mapillary_tools console messages

$ mapillary_tools video_process_and_upload ./temp/20231111_080922_NF.mp4 frames/ --video_sample_distance 0.5 --interpolate_directions --filetypes raw_blackvue --skip_process_errors --user_name "gowin" 2023-11-19 09:34:50,971 - INFO - Extracting video information: ffprobe -print_format json -hide_banner -show_format -show_streams temp/20231111_080922_NF.mp4 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55a0525f8c80] Found duplicated MOOV Atom. Skipped it Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'temp/20231111_080922_NF.mp4': Metadata: creation_time : 2023-11-11T00:10:23.000000Z major_brand : mp42 minor_version : 1 compatible_brands: mp42mp42 Duration: 00:01:00.98, start: 0.000000, bitrate: 25810 kb/s Stream #0:0(eng): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709), 3840x2160, 25172 kb/s, 29.99 fps, 30 tbr, 1k tbn, 1k tbc (default) Metadata: creation_time : 2023-11-11T00:10:23.000000Z handler_name : PittaSoft Video Media Handler vendor_id : [0][0][0][0] Stream #0:1(eng): Audio: mp3 (.mp3 / 0x33706D2E), 11025 Hz, mono, fltp, 16 kb/s (default) Metadata: creation_time : 2023-11-11T00:10:23.000000Z handler_name : PittaSoft Sound Media Handler vendor_id : [0][0][0][0] 2023-11-19 09:34:51,081 - INFO - Extracting video metdata 2023-11-19 09:34:51,084 - WARNING - Empty GPS data found 2023-11-19 09:34:51,084 - WARNING - Force the option "filetypes" to be "image" to avoid processing and uploading both the video samples and the videos themselves Validating metadatas: 0metadata [00:00, ?metadata/s] 2023-11-19 09:34:51,108 - INFO - Checking upload status for 0 metadatas 2023-11-19 09:34:51,401 - INFO - Uploading to organization: {"slug": "mapamore", "description": "#Nowhere2KnowWhere", "name": "MapAmore", "id": "839022377015250"} 2023-11-19 09:34:51,405 - INFO - Nothing uploaded. Bye.

govvin commented 7 months ago

As a workaround, I've had some success processing and uploading the files if I extract the track with exiftool first:

$ exiftool -ext mp4 -n -ee -api LargeFileSupport=1 -X temp/ > /tmp/exiftool_output.xml

$ mapillary_tools video_process temp/ frames/ --geotag_source exiftool --geotag_source_path /tmp/exiftool_output.xml --video_sample_distance -1 --video_sample_interval 0.5  --overwrite_all_EXIF_tags --num_processes 2 --skip_process_errors --rerun

$ mapillary_tools upload frames
ptpt commented 5 months ago

Hey @govvin the link to the sample video is expired. Could you try again?

The exiftool approach looks good to me.