Upload progress doesn't get properly recorded

ltog commented 5 years ago

I have some sequences of EXIF-tagged pictures I'd like to upload. For unknown reasons they were interrupted. (Message was: Failed to upload sequence at /path/to/pictures. Restart the script in order to finish you upload for this sequence. and also the thing described in https://github.com/openstreetcam/upload-scripts/issues/86 happened.) I'm not sure if the issue I'm going to describe is a consequence of that...

Whenever I start to upload the sequences the counter says 2605/9267 images. Altough the upload script uploaded gigabytes of data and eventually finished (again with Failed to upload sequence at /path/to/pictures. Restart the script in order to finish you upload for this sequence.), this progress doesn't seem to get recorded:

If I restart the upload of the sequences again, the counter says 2605/9267.
If I cancel the upload and restart again, the counter says 2605/9267.

What I would expect:

Even tough a sequence might not get uploaded completely in the first or second try, some progress should be made and be recorded in the sequence's progress data.
Ctrl-C-ing a running upload should not prevent progress data from being written.

Some infos:

I used version 600c3d4
Python version is 3.6.7 on Ubuntu

ltog commented 5 years ago

Addendum: The upload gets a lot this messages, when run with logging on debug level:

2019-06-09 13:36:53,042 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2651
2019-06-09 13:36:53,419 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2646
2019-06-09 13:36:53,716 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2652
2019-06-09 13:36:53,788 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2650
2019-06-09 13:36:53,970 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2645
2019-06-09 13:36:54,024 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2653
2019-06-09 13:36:54,179 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2649
2019-06-09 13:36:54,414 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2647
2019-06-09 13:36:54,523 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2648
2019-06-09 13:36:54,601 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2651
2019-06-09 13:36:54,703 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2646
2019-06-09 13:36:54,717 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2654
2019-06-09 13:36:54,945 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2652
2019-06-09 13:36:55,221 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2650
2019-06-09 13:36:55,363 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2653
2019-06-09 13:36:55,602 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2649
2019-06-09 13:36:55,653 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2645
2019-06-09 13:36:55,927 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2648
2019-06-09 13:36:55,964 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2647
2019-06-09 13:36:56,161 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2646
2019-06-09 13:36:56,673 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2654
2019-06-09 13:36:57,067 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2650
2019-06-09 13:36:57,174 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2652
2019-06-09 13:36:57,182 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2648
2019-06-09 13:36:57,199 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2651
2019-06-09 13:36:57,218 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2653
2019-06-09 13:36:57,233 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2649
2019-06-09 13:36:57,371 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2647
2019-06-09 13:36:57,387 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2646
2019-06-09 13:36:57,391 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2645
2019-06-09 13:36:58,267 osc_tools.osc_api_gateway          : DEBUG    Failed to upload photo index: 2654

mirunac-telenav commented 5 years ago

@ltog - an unfortunate storage downtime occurred and it took a while to get it resolved. Everything is up and running now. @bogdans-telenav - any reason why the upload should't work right now?

carstenhag commented 5 years ago

Bump -- Would be nice if issues were cleared up :)

salabogdan commented 1 year ago

Issue should be fixed in the meantime.

kartaview / upload-scripts

Upload progress doesn't get properly recorded #87