upload_photos_by_exif.py: Temporary connection glitch causes unhandled error

[ToeBee]

My wireless connection just blinked out for a few seconds while uploading. After it came back, the upload script started transferring again but did not print anything out on the console. I could see network traffic on the wireless interface and I could see more pictures being added by refreshing my profile page. But no "Uploaded - xx of total ..." messages were being printed.

Then after maybe 5 minutes the connection finally timed out and the whole script crashed out. If I'm reading this correctly, it appears that the script ended up crashing out because of a print message in an exception handling block.

[...]
Uploaded - 903 of total :1124, percentage: 80.34%

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    body=body, headers=headers)
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 353, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.4/http/client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.4/http/client.py", line 1126, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.4/http/client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.4/http/client.py", line 926, in _send_output
    self.send(message_body)
  File "/usr/lib/python3.4/http/client.py", line 888, in send
    self.sock.sendall(data)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/requests/adapters.py", line 376, in send
    timeout=timeout
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 609, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/util/retry.py", line 247, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/packages/six.py", line 309, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    body=body, headers=headers)
  File "/usr/local/lib/python3.4/dist-packages/requests/packages/urllib3/connectionpool.py", line 353, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.4/http/client.py", line 1088, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.4/http/client.py", line 1126, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.4/http/client.py", line 1084, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.4/http/client.py", line 926, in _send_output
    self.send(message_body)
  File "/usr/lib/python3.4/http/client.py", line 888, in send
    self.sock.sendall(data)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', timeout('timed out',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/toby/git/osv-uploader/upload_photos_by_exif/upload_photos_by_exif.py", line 122, in thread
    data = future.result()['json']
  File "/usr/lib/python3.4/concurrent/futures/_base.py", line 395, in result
    return self.__get_result()
  File "/usr/lib/python3.4/concurrent/futures/_base.py", line 354, in __get_result
    raise self._exception
  File "/usr/lib/python3.4/concurrent/futures/thread.py", line 54, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/toby/git/osv-uploader/upload_photos_by_exif/upload_photos_by_exif.py", line 112, in upload_photos
    conn = requests.post(url_photo, data=data_photo, files=photo, timeout=timeout)
  File "/usr/local/lib/python3.4/dist-packages/requests/api.py", line 107, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/adapters.py", line 426, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', timeout('timed out',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/toby/git/osv-uploader/upload_photos_by_exif/upload_photos_by_exif.py", line 406, in <module>
    main(sys.argv[1:])
  File "/home/toby/git/osv-uploader/upload_photos_by_exif/upload_photos_by_exif.py", line 387, in main
    count_uploaded = thread(max_workers, url_photo, list_to_upload, path, count_uploaded, total_img)
  File "/home/toby/git/osv-uploader/upload_photos_by_exif/upload_photos_by_exif.py", line 143, in thread
    print('%generated an exception: %s' % (exc))
TypeError: a float is required

[ToeBee]

Also, When the error happened, it wrote the number of the picture that errored out (number 903) to count_file.txt even though it had uploaded 100 more pictures (it was sitting at 1,001) by the time the timeout finally happened. So when I tried to resume the upload, the pictures kept getting rejected by the server as duplicates. Eventually I figured out what was happening and modified count_file.txt to say 1000 to skip ahead.

This behavior might be ok though since you don't want to miss images in an upload.

[bogdan-racasan]

Thanks for reporting 👍. Test it again and if there is a problem open again this issue. Thanks again