OpenDroneMap / ODM

A command line toolkit to generate maps, point clouds, 3D models and DEMs from drone, balloon or kite images. 📷
https://opendronemap.org
GNU Affero General Public License v3.0
4.89k stars 1.11k forks source link

photo.py fails to read EXIF information of Chinese camera manufacturer #1120

Closed copterspace closed 4 years ago

copterspace commented 4 years ago

How did you install OpenDroneMap? (Docker, natively, ...)?

Docker on Ubuntu 18.04.4 LTS, according to WebODM getting started Readme.

What's your browser and operating system?

Firefox 77 on Windows 10 (issue is not related to the browser)

What is the problem?

I uploaded 50 photos from Xiaomi 4K drone to the system, tried to process them with default options. Short after I've got an error message in the Dashboard task output:

File "/code/opendm/photo.py", line 87, in parse_exif_values
self.camera_make = tags['Image Make'].values.encode('utf8')
UnicodeDecodeError
: 'ascii' codec can't decode byte 0xb1 in position 0: ordinal not in range(128)

What should be the expected behavior?

Photo.py should not fail because of strange Chinese symbols in EXIF. I changed manually Camera Manufacturer in all my files from "±±ѕ©·ЙГЧїЖјјУРПЮ№«Лѕ" to "Xiaomi" - now the processing does not fall out with this error.

How can we reproduce this?

Please see I attach one original photo with Chineese camera manufacturer field (packed to ZIP to preserve all info). IMG_20170101_010153.zip I think the shortest way is to process this file with photo.py script directly - you should see the same error (rather then feed it all way through docker/WebODM stack). I think some extra error handling could be easely added to photo.py - but I'm only started to test WebODM, not sure I can fix it myself fast enough to send pull request + original developers sight on this issue is important. Currently, manual correction of EXIF camera manufacturer value helped me - but it changed the original files.

Bellow is full dashboard task output:

[INFO]    Initializing ODM - Mon Jun 15 13:25:49  2020
[INFO]    ==============
[INFO]    build_overviews: False
[INFO]    camera_lens: auto
[INFO]    cameras: {}
[INFO]    crop: 3
[INFO]    debug: False
[INFO]    dem_decimation: 1
[INFO]    dem_euclidean_map: False
[INFO]    dem_gapfill_steps: 3
[INFO]    dem_resolution: 5
[INFO]    depthmap_resolution: 1000.0
[INFO]    dsm: False
[INFO]    dtm: False
[INFO]    end_with: odm_report
[INFO]    fast_orthophoto: False
[INFO]    feature_type: sift
[INFO]    force_gps: False
[INFO]    gcp: None
[INFO]    gps_accuracy: 15
[INFO]    ignore_gsd: False
[INFO]    matcher_distance: 0
[INFO]    matcher_neighbors: 8
[INFO]    max_concurrency: 8
[INFO]    merge: all
[INFO]    mesh_octree_depth: 11
[INFO]    mesh_point_weight: 4
[INFO]    mesh_samples: 1.0
[INFO]    mesh_size: 300000
[INFO]    min_num_features: 8000
[INFO]    mve_confidence: 0.6
[INFO]    name: 85cfce79-029e-4242-a530-b96240c3bc85
[INFO]    opensfm_depthmap_method: PATCH_MATCH
[INFO]    opensfm_depthmap_min_consistent_views: 3
[INFO]    opensfm_depthmap_min_patch_sd: 1
[INFO]    optimize_disk_space: False
[INFO]    orthophoto_compression: DEFLATE
[INFO]    orthophoto_cutline: False
[INFO]    orthophoto_no_tiled: False
[INFO]    orthophoto_png: False
[INFO]    orthophoto_resolution: 5
[INFO]    pc_classify: False
[INFO]    pc_csv: False
[INFO]    pc_ept: False
[INFO]    pc_filter: 2.5
[INFO]    pc_las: False
[INFO]    pc_rectify: False
[INFO]    pc_sample: 0
[INFO]    project_path: /var/www/data
[INFO]    radiometric_calibration: none
[INFO]    rerun: None
[INFO]    rerun_all: False
[INFO]    rerun_from: ['dataset', 'split', 'merge', 'opensfm', 'mve', 'odm_filterpoints', 'odm_meshing', 'mvs_texturing', 'odm_georeferencing', 'odm_dem', 'odm_orthophoto', 'odm_report']
[INFO]    resize_to: 2048
[INFO]    skip_3dmodel: False
[INFO]    sm_cluster: None
[INFO]    smrf_scalar: 1.25
[INFO]    smrf_slope: 0.15
[INFO]    smrf_threshold: 0.5
[INFO]    smrf_window: 18.0
[INFO]    split: 999999
[INFO]    split_multitracks: False
[INFO]    split_overlap: 150
[INFO]    texturing_data_term: gmi
[INFO]    texturing_keep_unseen_faces: False
[INFO]    texturing_nadir_weight: 16
[INFO]    texturing_outlier_removal_type: gauss_clamping
[INFO]    texturing_skip_global_seam_leveling: False
[INFO]    texturing_skip_hole_filling: False
[INFO]    texturing_skip_local_seam_leveling: False
[INFO]    texturing_skip_visibility_test: False
[INFO]    texturing_tone_mapping: none
[INFO]    time: False
[INFO]    use_3dmesh: True
[INFO]    use_exif: False
[INFO]    use_fixed_camera_params: False
[INFO]    use_hybrid_bundle_adjustment: False
[INFO]    use_opensfm_dense: False
[INFO]    verbose: False
[INFO]    ==============
[INFO]    Running dataset stage
[INFO]    Loading dataset from: /var/www/data/85cfce79-029e-4242-a530-b96240c3bc85/images
[INFO]    Loading 50 images
Traceback (most recent call last):
File "/code/run.py", line 61, in <module>
app.execute()
File "/code/stages/odm_app.py", line 95, in execute
self.first_stage.run()
File "/code/opendm/types.py", line 331, in run
self.process(self.args, outputs)
File "/code/stages/dataset.py", line 89, in process
photos += [types.ODM_Photo(f)]
File "/code/opendm/photo.py", line 68, in __init__
self.parse_exif_values(path_file)
File "/code/opendm/photo.py", line 87, in parse_exif_values
self.camera_make = tags['Image Make'].values.encode('utf8')
UnicodeDecodeError
: 'ascii' codec can't decode byte 0xb1 in position 0: ordinal not in range(128)
pierotofy commented 4 years ago

Thanks for the report (and test image) @copterspace ! :pray:

This should have been fixed with #1122. As soon as the docker images are rebuilt, try to update and process again. It should work. If not, please re-open this issue. :+1: