search

TheFox / flickr-cli

A command-line interface to Flickr. Upload and download photos, photo sets, directories via shell.
https://fox21.at
25 stars 9 forks source link

flickr-cli does not repeat failed upload #30

Open ruzickap opened 6 years ago

ruzickap commented 6 years ago

Hello.

I tried to upload some batch of images to Flickr and unfortunately I got API error (which happens from time to time), but the flickr-cli didn't repeat the upload.

Here is the error log:

$ docker run --rm -it -u 1000:1000 -v /home/pruzicka/data/_Photos_/\!DVD1:/mnt:ro -v /home/pruzicka/.flickr-cli:/data:ro thefox21/flickr-cli upload --config=/data/config.yml --tags "2017.12.02 Drazdany_Moritzburg" --sets "2017.12.02-Drazdany_Moritzburg" 2017.12.02-Drazdany_Moritzburg
[file] 100.00% [###################################] 11,361,268   1.89MB/s
[file] 100.00% [###################################] 13,043,866   2.17MB/s
[file] 100.00% [###################################] 13,417,716   2.24MB/s
[file] 100.00% [###################################] 14,110,620   2.35MB/s
[file] 100.00% [###################################] 13,779,415   2.30MB/s
[file] 100.00% [###################################] 11,674,896   1.95MB/s
[file] 100.00% [###################################] 13,781,063   2.30MB/s
[2017-12-15 14:29:44] upload.ERROR: [file] upload: Server error response [status code] 500 [reason phrase] Internal Server Error [url] https://up.flickr.com/services/upload/ [] []
[file] 100.00% [###################################] 12,646,758   2.11MB/s
[file] 100.00% [###################################] 11,949,811   1.99MB/s

It means that the file which ended with error 500 was not uploaded and flickr-cli didn't repeat the upload process.

It would be nice if flickr-cli can repeat the failed uploads.

Thank you...

ruzickap commented 6 years ago

Here is example executed with "-vvv" 

[2017-12-15 15:36:23] upload.INFO: [file] upload "20171028-163226.jpg" 13.37MB [] []
[file] 100.00% [###################################] 13,367,926   2.23MB/s
[2017-12-15 15:36:28] upload.INFO: [file] status: OK - ID 39040884372 [] []
[2017-12-15 15:36:28] upload.INFO: [file] add to sets ...  [] []
[2017-12-15 15:36:29] upload.CRITICAL: [file] add to sets FAILED: Server error response [status code] 500 [reason phrase] Internal Server Error [url] https://api.flickr.com/services/rest/ [] []
ruzickap commented 6 years ago

For example this error which I got from Flickr completely stoped the uploading process:

...
...
[2017-12-15 16:21:57] upload.CRITICAL: [file] add to sets FAILED: Server error response [status code] 502 [reason phrase] Cannot find server. [url] https://api.flickr.com/services/rest/ [] []
$
TheFox commented 6 years ago

In the old version this was catched by a separate logger. But this solution was too complicated. I don't know if a retry is a good idea. I don't want to stress the Flickr API. Are you expecting a retry?

ruzickap commented 6 years ago

My experience with Flickr API is that form time to time it gets such errors, but when you try it again (reupload the file / redo the action) it works fine.

What about retry the failed operation three times and then let flickr-cli die with the error? -> That should not stress Flickr API so much.