kartaview / upload-scripts

Uploader tools for KartaView
MIT License
59 stars 30 forks source link

No sequence found. #107

Closed joshinils closed 1 year ago

joshinils commented 3 years ago

I want to upload images which i have taken with the mapillary app. these images are now on my hard drive, they come off an iphone 8. this is the output i get when trying to upload the folder:

➜  ~ python3 ~/Documents/upload-scripts/osc_tools.py upload -p ~/Pictures/2021-01-18
Searching for sequences...
Search completed.
    No sequence found.

this is the exif:

➜  ~ exiftool ~/Pictures/2021-01-18/IMG_5803.JPG 
ExifTool Version Number         : 11.88
File Name                       : IMG_5803.JPG
Directory                       : /home/niels/Pictures/2021-01-18
File Size                       : 2.8 MB
File Modification Date/Time     : 2021:01:18 12:49:30+01:00
File Access Date/Time           : 2021:04:08 16:39:10+02:00
File Inode Change Date/Time     : 2021:01:18 14:28:36+01:00
File Permissions                : rw-rw-r--
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
Exif Byte Order                 : Big-endian (Motorola, MM)
Image Description               : {"MAPCompassHeading":{"TrueHeading":256.78939640345874,"MagneticHeading":256.78939640345874,"AccuracyDegrees":0},"MAPGpsTime":"2021_01_18_11_49_30_143","MAPVersionString":"4.20.3 (380)","MAPLatitude":52.631939227776719,"MAPCaptureTime":"2021_01_18_11_49_30_410","MAPGPSSpeed":9.5185184478759766,"MAPDeviceModel":"iPhone 8","MAPDeviceUUID":"A661877E-2291-4EAB-BF41-381709F16FD3","MAPSettingsUserKey":"EnKnkPu35HGG6eUyAGkM7E","MAPAppNameString":"mapillary_ios","MAPSettingsTokenValid":1,"MAPAltitude":54.802884486654136,"MAPLocalTimeZone":"Europe\/Berlin (MEZ) offset 3600","MAPGPSAccuracyMeters":8.0012082736808772,"MAPAtanAngle":-3.1274178028106689,"MAPPhotoUUID":"948FF486-F691-44CB-9B2C-C585769BFF95","MAPLongitude":13.295124552877734,"MAPSettingsUploadHash":"e821976ae7dea1115e8c4065bcd35d83f20c6996550d2f3f2de4b0e8456b0727","MAPDeviceMake":"Apple","MAPSequenceUUID":"51A8F541-A695-4364-8842-0CC6FA744424","MAPAccelerometerVector":{"x":-0.99921602010726929,"y":-0.01416475884616375,"z":0.036962460726499557}}
Make                            : Apple
Camera Model Name               : iPhone 8
Orientation                     : Horizontal (normal)
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Exposure Time                   : 1/100
F Number                        : 1.8
Exposure Program                : Program AE
ISO                             : 25
Exif Version                    : 0231
Offset Time                     : +01:00
Offset Time Original            : +01:00
Offset Time Digitized           : +01:00
Components Configuration        : Y, Cb, Cr, -
Shutter Speed Value             : 1/100
Aperture Value                  : 1.8
Brightness Value                : 5.640695805
Exposure Compensation           : 0
Metering Mode                   : Multi-segment
Focal Length                    : 4.0 mm
Subject Area                    : 2015 1511 2217 1330
Sub Sec Time Original           : 399
Sub Sec Time Digitized          : 399
Flashpix Version                : 0100
Color Space                     : Uncalibrated
Exif Image Width                : 4032
Exif Image Height               : 3024
Sensing Method                  : One-chip color area
Scene Type                      : Directly photographed
Exposure Mode                   : Auto
White Balance                   : Auto
Focal Length In 35mm Format     : 28 mm
Scene Capture Type              : Standard
Lens Info                       : 3.99000001mm f/1.8
Lens Make                       : Apple
Lens Model                      : iPhone 8 back camera 3.99mm f/1.8
GPS Altitude                    : 54 m
GPS Dilution Of Precision       : 8
GPS Speed                       : 9
GPS Img Direction               : 256
XMP Toolkit                     : XMP Core 6.0.0
GPS Date/Time                   : 11:49:29.958000
Flash Pix Version               : 0100
Date/Time Original              : 2021:01:18 11:49:30
Subsec Time Original            : 399
GPS Date Stamp                  : 2021:01:18
Subsec Time Digitized           : 399
Focal Len In 35mm Film          : 28
Tag Y Cb Cr Positioning         : 1
Photographic Sensitivity        : 25
Modify Date                     : 2021:01:18 11:49:30
Description                     : {"MAPCompassHeading":{"TrueHeading":256.78939640345874,"MagneticHeading":256.78939640345874,"AccuracyDegrees":0},"MAPGpsTime":"2021_01_18_11_49_30_143","MAPVersionString":"4.20.3 (380)","MAPLatitude":52.631939227776719,"MAPCaptureTime":"2021_01_18_11_49_30_410","MAPGPSSpeed":9.5185184478759766,"MAPDeviceModel":"iPhone 8","MAPDeviceUUID":"A661877E-2291-4EAB-BF41-381709F16FD3","MAPSettingsUserKey":"EnKnkPu35HGG6eUyAGkM7E","MAPAppNameString":"mapillary_ios","MAPSettingsTokenValid":1,"MAPAltitude":54.802884486654136,"MAPLocalTimeZone":"Europe\/Berlin (MEZ) offset 3600","MAPGPSAccuracyMeters":8.0012082736808772,"MAPAtanAngle":-3.1274178028106689,"MAPPhotoUUID":"948FF486-F691-44CB-9B2C-C585769BFF95","MAPLongitude":13.295124552877734,"MAPSettingsUploadHash":"e821976ae7dea1115e8c4065bcd35d83f20c6996550d2f3f2de4b0e8456b0727","MAPDeviceMake":"Apple","MAPSequenceUUID":"51A8F541-A695-4364-8842-0CC6FA744424","MAPAccelerometerVector":{"x":-0.99921602010726929,"y":-0.01416475884616375,"z":0.036962460726499557}}
Flash Function                  : False
Flash Fired                     : False
Flash Return                    : No return detection
Flash Mode                      : Off
Flash Red Eye Mode              : False
Profile CMM Type                : Apple Computer Inc.
Profile Version                 : 4.0.0
Profile Class                   : Display Device Profile
Color Space Data                : RGB
Profile Connection Space        : XYZ
Profile Date Time               : 2017:07:07 13:22:32
Profile File Signature          : acsp
Primary Platform                : Apple Computer Inc.
CMM Flags                       : Not Embedded, Independent
Device Manufacturer             : Apple Computer Inc.
Device Model                    : 
Device Attributes               : Reflective, Glossy, Positive, Color
Rendering Intent                : Perceptual
Connection Space Illuminant     : 0.9642 1 0.82491
Profile Creator                 : Apple Computer Inc.
Profile ID                      : ca1a9582257f104d389913d5d1ea1582
Profile Description             : Display P3
Profile Copyright               : Copyright Apple Inc., 2017
Media White Point               : 0.95045 1 1.08905
Red Matrix Column               : 0.51512 0.2412 -0.00105
Green Matrix Column             : 0.29198 0.69225 0.04189
Blue Matrix Column              : 0.1571 0.06657 0.78407
Red Tone Reproduction Curve     : (Binary data 32 bytes, use -b option to extract)
Chromatic Adaptation            : 1.04788 0.02292 -0.0502 0.02959 0.99048 -0.01706 -0.00923 0.01508 0.75168
Blue Tone Reproduction Curve    : (Binary data 32 bytes, use -b option to extract)
Green Tone Reproduction Curve   : (Binary data 32 bytes, use -b option to extract)
Image Width                     : 4032
Image Height                    : 3024
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Aperture                        : 1.8
Image Size                      : 4032x3024
Megapixels                      : 12.2
Scale Factor To 35 mm Equivalent: 7.0
Shutter Speed                   : 1/100
GPS Latitude                    : 52 deg 37' 54.98" N
GPS Longitude                   : 13 deg 17' 42.45" E
Flash                           : Off, Did not fire
GPS Latitude Ref                : North
GPS Longitude Ref               : East
Circle Of Confusion             : 0.004 mm
Field Of View                   : 65.5 deg
Focal Length                    : 4.0 mm (35 mm equivalent: 28.0 mm)
GPS Position                    : 52 deg 37' 54.98" N, 13 deg 17' 42.45" E
Hyperfocal Distance             : 2.07 m
Light Value                     : 10.3

the long description is what is generated by mapillary.

i want to upload these to kartaview aswell, since some sequences just dont show up on mapillary, or not in full (which is weird, and i dont like it). in future i may just upload to kartaview, not to mapillary anymore, i have not yet tried the kartaview app to take images.

BeJotEl commented 3 years ago

Found a workaround to upload mapillary generated images: (re)add Date/Time Original from original filenames (e.g. 2020_04_02_06_30_03_519.jpg) exiftool -recurse "-datetimeoriginal<filename" -overwrite_original -progress -preserve -ext jpg -fast2 /path/to/images

Afterwards upload is possible without hassle

erias04 commented 2 years ago

When I run that exiftool command from @BeJotEl, it gives me this output:

> /IMG_2583_000467.jpg
> ======== /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000468.jpg [463/467]
> Warning: Month '00' out of range 1..12 in ExifIFD:DateTimeOriginal (PrintConvInv) - /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000468.jpg
> Warning: No writable tags set from /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000468.jpg
> Nothing changed in /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000468.jpg
> ======== /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000469.jpg [464/467]
> Warning: Month '00' out of range 1..12 in ExifIFD:DateTimeOriginal (PrintConvInv) - /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000469.jpg
> Warning: No writable tags set from /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000469.jpg
> Nothing changed in /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000469.jpg
> ======== /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000470.jpg [465/467]
> Warning: Month '00' out of range 1..12 in ExifIFD:DateTimeOriginal (PrintConvInv) - /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000470.jpg
> Warning: No writable tags set from /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000470.jpg
> Nothing changed in /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000470.jpg
> ======== /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000471.jpg [466/467]
> Warning: Month '00' out of range 1..12 in ExifIFD:DateTimeOriginal (PrintConvInv) - /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000471.jpg
> Warning: No writable tags set from /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000471.jpg
> Nothing changed in /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000471.jpg
> ======== /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000472.jpg [467/467]
> Warning: Month '00' out of range 1..12 in ExifIFD:DateTimeOriginal (PrintConvInv) - /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000472.jpg
> Warning: No writable tags set from /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000472.jpg
> Nothing changed in /home/erias/Documents/mapillary/test2/extractedImages/mapillary_sampled_video_frames/IMG_2583/IMG_2583_000472.jpg
> 1 directories scanned
> 0 image files updated
> 467 image files unchanged

So it is not working for me :(

Is there any other way to upload it? (These pictures are sampled by mapillary and they have a GPS metatag. So they have all requirements, right?)

Not even directly upload the images with GPS-tags works with: python osc_tools.py upload -p /path/to/images. (I am using python3, Python 3.9.6)

I am looking forward for a reply.

This is one of the images: (Download it to view the metadata) IMG_2580 1_000005

BeJotEl commented 2 years ago

Sorry for being unclear my command works only for files with the original filename exported from iPhone e.g 2020_04_02_06_30_03_519.jpg because the date/time information is created from this name, date: 2020-04-02 at 6:30, 3 seconds and 519 microseconds so if you change the filename to something else it will not work

salabogdan commented 2 years ago

Sorry guys for the late replays. I will look into this in the next couple of days.

salabogdan commented 2 years ago

I just now had some time to read all the posts and understand the problem. This is nice! Let me make some changes to add support for this type of images. Basically read the image description and get the required information from there.

lionkmp commented 2 years ago

@salabogdan I'm curious what information is missing from this exif? Is it impossible to upload images taken with Mapillary App?

salabogdan commented 1 year ago

Long story short, Mapillary adds some custom headers to the exif, and we need to read them in order to have the info.
I investigated this back then but forgot to push the branch and now I no longer have it in the computer. This is what I am taking about. {"MAPCompassHeading":{"TrueHeading":256.78939640345874,"MagneticHeading":256.78939640345874,"AccuracyDegrees":0},"MAPGpsTime":"2021_01_18_11_49_30_143","MAPVersionString":"4.20.3 (380)","MAPLatitude":52.631939227776719,"MAPCaptureTime":"2021_01_18_11_49_30_410","MAPGPSSpeed":9.5185184478759766,"MAPDeviceModel":"iPhone 8","MAPDeviceUUID":"A661877E-2291-4EAB-BF41-381709F16FD3","MAPSettingsUserKey":"EnKnkPu35HGG6eUyAGkM7E","MAPAppNameString":"mapillary_ios","MAPSettingsTokenValid":1,"MAPAltitude":54.802884486654136,"MAPLocalTimeZone":"Europe\/Berlin (MEZ) offset 3600","MAPGPSAccuracyMeters":8.0012082736808772,"MAPAtanAngle":-3.1274178028106689,"MAPPhotoUUID":"948FF486-F691-44CB-9B2C-C585769BFF95","MAPLongitude":13.295124552877734,"MAPSettingsUploadHash":"e821976ae7dea1115e8c4065bcd35d83f20c6996550d2f3f2de4b0e8456b0727","MAPDeviceMake":"Apple","MAPSequenceUUID":"51A8F541-A695-4364-8842-0CC6FA744424","MAPAccelerometerVector":{"x":-0.99921602010726929,"y":-0.01416475884616375,"z":0.036962460726499557}} Should be easy to re-do the patch since I made some improvements on the part, let me see later today.