upload_photos_from_exif stops

[roirobo]

I tried twice and the upload stops, I think because of some json decoder error.

I'm using python3.5 and had problems before installing Pillow (it solved once I tried to upload example photos)

I get the this message:

/ Found 199 pictures to upload 64%|███████████████████████████████████████████████████████████████████▋ 127/199 remaining:12:30 elapsed:18:45multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/usr/lib/python3.5/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, kwds)) File "/home/nomada/MapillaryTools/upload-scripts/upload_photos_by_exif/photo.py", line 79, in call self.make_upload(photo, data_photo) File "~/MapillaryTools/upload-scripts/upload_photos_by_exif/photo.py", line 60, in make_upload if conn.json()['status']['apiMessage'] == ' You are not allowed to add a duplicate entry (sequenceIndex)': File "~/MapillaryTools/upload-scripts/lib/python3.5/site-packages/requests/models.py", line 808, in json return complexjson.loads(self.text, kwargs) File "/usr/lib/python3.5/json/init.py", line 319, in loads return _default_decoder.decode(s) File "/usr/lib/python3.5/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.5/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) """

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "upload_exif.py", line 67, in main() File "upload_exif.py", line 58, in main do_upload(thread, UploadPhoto(path, access_token, id_sequence, count_list, url_photo), photos_path) File "~/MapillaryTools/upload-scripts/osc/utils.py", line 67, in do_upload list(tqdm.tqdm(p.imap(generator, payload), total=len(payload), bar_format='{l_bar}{bar} {n_fmt}/{total_fmt} remaining:{remaining} elapsed:{elapsed}')) File "~/MapillaryTools/upload-scripts/lib/python3.5/site-packages/tqdm/_tqdm.py", line 872, in iter for obj in iterable: File "/usr/lib/python3.5/multiprocessing/pool.py", line 695, in next raise value json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[alexandrui-telenav]

Could you update to Python 3.6 and retry? I'm not very familiar with the script's implementation but asked around and that was the advice.

[ToeBee]

I have been seeing steady upload failures as well with the same error message. I can rarely get more than a couple hundred pictures uploaded at a time. I find it unlikely that this is a python version problem. Note the error point is on line 60 of photo.py. It can only get to this location if the server returns a non-200 status code.

So the server is encountering some kind of error while processing a picture file. Due to this error it is not returning valid JSON to the upload script. This causes the conn.json() function to blow up because it can't parse the string. I just added debugging in this location to see if I can get exactly what the server is returning. Now watch it finish 5,000 pictures without error just to annoy me...

[ToeBee]

Moving discussion to openstreetcam/openstreetview.org#157 because this is definitely a server side error.

[danbjoseph]

accidentally additionally logged https://github.com/openstreetcam/upload-scripts/issues/51 about this problem (i'm using Python 3.6.5)

[rwelty1889]

just got this message on my second upload using the script, uncertain how to proceed.

[alexandrui-telenav]

Please retry, the script should only attempt to upload new files

[bogdans-telenav]

Update: Started work on a new upload script. After making sure that the new one will fix this i will close this issue.

[bogdans-telenav]

Finally done it! Thanks for your patience! If you find any issues with the new version please create a issue and we will address it as soon as possible.