search

kartaview / upload-scripts

Uploader tools for KartaView
MIT License
61 stars 30 forks source link

Fails to upload with "list index out of range" in line _timestamp.values[1] of exif.py #140

Open Talkless opened 6 months ago

Talkless commented 6 months ago

exiftool dump of picture in quesiton:

---- ExifTool ----
ExifTool Version Number         : 12.76
---- System ----
File Name                       : image20240303_101153395_fixed.jpg
Directory                       : .
File Size                       : 1882 kB
File Modification Date/Time     : 2024:03:10 19:37:01+02:00
File Access Date/Time           : 2024:03:10 19:37:01+02:00
File Inode Change Date/Time     : 2024:03:10 20:21:52+02:00
File Permissions                : -rw-rw-r--
---- File ----
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
Exif Byte Order                 : Big-endian (Motorola, MM)
Image Width                     : 4096
Image Height                    : 2304
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
---- JFIF ----
JFIF Version                    : 1.01
Resolution Unit                 : None
X Resolution                    : 1
Y Resolution                    : 1
---- IFD0 ----
Image Description               : 
Make                            : Volla
Camera Model Name               : Volla Phone X
Orientation                     : Horizontal (normal)
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Software                        : MediaTek Camera Application
Modify Date                     : 2024:03:03 08:11:53
Y Cb Cr Positioning             : Co-sited
---- ExifIFD ----
Exposure Time                   : 1/33
F Number                        : 2.0
Exposure Program                : Not Defined
ISO                             : 162
Exif Version                    : 0220
Date/Time Original              : 2024:03:03 10:11:53
Create Date                     : 2024:03:03 10:11:53
Components Configuration        : Y, Cb, Cr, -
Exposure Compensation           : 0
Metering Mode                   : Center-weighted average
Light Source                    : Other
Flash                           : No Flash
Focal Length                    : 3.5 mm
Sub Sec Time                    : 2
Sub Sec Time Original           : 2
Sub Sec Time Digitized          : 2
Flashpix Version                : 0100
Color Space                     : sRGB
Exif Image Width                : 4096
Exif Image Height               : 2304
Exposure Mode                   : Auto
White Balance                   : Auto
Digital Zoom Ratio              : 1
Focal Length In 35mm Format     : 20 mm
Scene Capture Type              : Standard
---- InteropIFD ----
Interoperability Index          : R98 - DCF basic file (sRGB)
---- GPS ----
GPS Version ID                  : 2.2.0.0
GPS Latitude Ref                : North
GPS Latitude                    : 55 deg 49' 30.41"
GPS Longitude Ref               : East
GPS Longitude                   : 23 deg 4' 50.38"
GPS Altitude Ref                : Above Sea Level
GPS Altitude                    : 148 m
GPS Time Stamp                  : 346409:07:30
GPS Processing Method           : GPS
GPS Date Stamp                  : 
---- IFD1 ----
Compression                     : JPEG (old-style)
Orientation                     : Horizontal (normal)
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Y Cb Cr Positioning             : Co-sited
Thumbnail Offset                : 1075
Thumbnail Length                : 27630
Thumbnail Image                 : (Binary data 27630 bytes, use -b option to extract)
---- Composite ----
Aperture                        : 2.0
Image Size                      : 4096x2304
Megapixels                      : 9.4
Scale Factor To 35 mm Equivalent: 5.7
Shutter Speed                   : 1/33
Create Date                     : 2024:03:03 10:11:53.2
Date/Time Original              : 2024:03:03 10:11:53.2
Modify Date                     : 2024:03:03 08:11:53.2
GPS Altitude                    : 148 m Above Sea Level
GPS Date/Time                   :  346409:07:30Z
GPS Latitude                    : 55 deg 49' 30.41" N
GPS Longitude                   : 23 deg 4' 50.38" E
Circle Of Confusion             : 0.005 mm
Field Of View                   : 84.0 deg
Focal Length                    : 3.5 mm (35 mm equivalent: 20.0 mm)
GPS Position                    : 55 deg 49' 30.41" N, 23 deg 4' 50.38" E
Hyperfocal Distance             : 1.17 m
Light Value                     : 6.4

I use this wrapper script for performing uploads:

#!/usr/bin/bash

set -eu

readonly SCRIPT_DIR=$(readlink -f $(dirname "${BASH_SOURCE}"))
readonly VENV_DIR=${SCRIPT_DIR}/upload-scripts.git/venv

if [[ ! -d "${VENV_DIR}" ]]
then
    echo "Creating Python venv.."
    python3 -m venv "${VENV_DIR}"
fi

source "${SCRIPT_DIR}/upload-scripts.git/venv/bin/activate"
python3 -m ensurepip
python3 -m pip install --upgrade pip
pip install -r "${SCRIPT_DIR}/upload-scripts.git/requirements.txt"

echo "Uploading photos to Kartaview in "$(pwd)" directory..."
${SCRIPT_DIR}/upload-scripts.git/osc_tools.py upload  --path "$(pwd)"
echo "Done."

And this is result of executing this script:

Looking in links: /tmp/tmpkz0ywy8t
Requirement already satisfied: setuptools in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (68.1.2)
Requirement already satisfied: pip in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (24.0)
Requirement already satisfied: pip in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (24.0)
Requirement already satisfied: requests>=2.20.0 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from -r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 1)) (2.31.0)
Requirement already satisfied: ExifRead>=2.3.2 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from -r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 2)) (3.0.0)
Requirement already satisfied: tqdm>=4.62.3 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from -r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 3)) (4.66.2)
Requirement already satisfied: piexif>=1.1.3 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from -r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 4)) (1.1.3)
Requirement already satisfied: pycodestyle>=2.8.0 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from -r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 5)) (2.11.1)
Requirement already satisfied: gpxpy>=1.4.2 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from -r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 7)) (1.6.2)
Requirement already satisfied: geojson~=2.5.0 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from -r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 8)) (2.5.0)
Requirement already satisfied: requests-oauthlib in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from -r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 10)) (1.3.1)
Requirement already satisfied: imagesize~=1.3.0 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from -r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 13)) (1.3.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from requests>=2.20.0->-r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 1)) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from requests>=2.20.0->-r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 1)) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from requests>=2.20.0->-r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 1)) (2.2.1)
Requirement already satisfied: certifi>=2017.4.17 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from requests>=2.20.0->-r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 1)) (2024.2.2)
Requirement already satisfied: oauthlib>=3.0.0 in /home/vincas/opt/kartaview/upload-scripts.git/venv/lib/python3.11/site-packages (from requests-oauthlib->-r /home/vincas/opt/kartaview/upload-scripts.git/requirements.txt (line 10)) (3.2.2)
Uploading photos to Kartaview in /home/vincas/Darbastalis/TODO dirvonai/medinčius/fixed directory...
Searching for sequences...
Traceback (most recent call last):
  File "/home/vincas/opt/kartaview/upload-scripts.git/osc_tools.py", line 243, in <module>
    main()
  File "/home/vincas/opt/kartaview/upload-scripts.git/osc_tools.py", line 24, in main
    args.func(args)
  File "/home/vincas/opt/kartaview/upload-scripts.git/osc_tools.py", line 95, in upload_command
    sequences = discoverer.discover(path)
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vincas/opt/kartaview/upload-scripts.git/osc_discoverer.py", line 114, in discover
    sequence = self.create_sequence(path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vincas/opt/kartaview/upload-scripts.git/osc_discoverer.py", line 129, in create_sequence
    (visual_data, data_type) = self.visual_data.discover(path)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vincas/opt/kartaview/upload-scripts.git/visual_data_discover.py", line 50, in discover
    photo = cls._photo_from_path(os.path.join(path, file_path))
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vincas/opt/kartaview/upload-scripts.git/visual_data_discover.py", line 81, in _photo_from_path
    exif_parser.next_item_with_class(PhotoMetadata))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vincas/opt/kartaview/upload-scripts.git/parsers/exif/exif.py", line 47, in next_item_with_class
    return self._photo_item(self.tags)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vincas/opt/kartaview/upload-scripts.git/parsers/exif/exif.py", line 171, in _photo_item
    gps = self._gps_item(tags_data)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vincas/opt/kartaview/upload-scripts.git/parsers/exif/exif.py", line 133, in _gps_item
    gps.timestamp = self._gps_timestamp(tags_data)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vincas/opt/kartaview/upload-scripts.git/parsers/exif/exif.py", line 245, in _gps_timestamp
    minutes: Ratio = _timestamp.values[1]
                     ~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

Mapillary's uploaded accepted this photo successfully.

Maybe it's something to do with Debian Sid's transition to 64bit timestamps..?

robhubi commented 2 months ago

Which Python version did you use?

Talkless commented 2 months ago

@robhubi I'm on Debian Sid, it's Python 3.12 now.

robhubi commented 2 months ago

I'm on Windows with Python 3.12. I think our issues (#140 and #144) describe the same thing.