search

perkeep / gphotos-cdp

This program uses the Chrome DevTools Protocol to drive a Chrome session that downloads your photos stored in Google Photos.
Apache License 2.0
656 stars 36 forks source link

.lastdone removed without being replaced #23

Closed FrankReh closed 4 years ago

FrankReh commented 4 years ago

Thanks for a very useful tool.

Sorry if this kind of advice is elsewhere on the site already.

More of a warning to anyone that runs into the problem I did regarding the .lastdone file. (Put a copy in another directory before a restart.)

A tl;dr summary, when you are about to fire up a new run of the tool, copy the .lastdone from the downloads directory to a safe place in case you want to put it back before a subsequent restart.

The .lastdone file comes in very handy for sessions that are going to be long and that could be interrupted and therefore require a manual restart. This happened to me yesterday and again today.

The initial session can take a very long time to work its way from the last photo or video to the first (in my case about ten year's worth of images). Once it finds the first, it begins downloading and writing/rewriting a .lastdone file in the chosen download directory with the URL of each successive successful download.

Having a .lastdone saves time in two ways, it lets the tool avoid starting from the end and having to scroll its way backwards to the beginning, and it lets the tool avoid redownloading files from the beginning that it had already downloaded successfully.

Any number of things can cause the tool to have to exit. In my case, the one minute timeout for a download was exceeded (caused by a 240MB video). I found the Chrome window still open and on my Mac, closed it using the red button, had a look at the .lastdone file (only because I had looked through the code earlier) and then started the command again with the same arguments (-v -dev -dldir="$HOME/dldir"). But because the same Chrome process was still running, the tool wasn't allowed to start a new test version of Chrome and it aborted immediately with an appropriate log message. (On a mac, closing the window isn't the same as quitting the process, I was reminded.) But the bad part is the .lastdone file had been removed. There is a cleanup of all non-directory entries in the download directory that seems to get triggered even in the case when the chrome process is not going to be started successfully. Restarting the tool showed it starting from scratch.

I had the contexts of the last .lastdone file still in a terminal so I could manually create it to let the tool start from the problem video again. Since then, I copy the .lastdone to a temp directory before any restart.

mpl commented 4 years ago

duplicate of #20 maybe?

btw, an alternative to the .lastdone file is to use the -start flag.

FrankReh commented 4 years ago

20 ends up referring to a TODO about making a backup, or moving, before writing a new file but my case was that all the non-directory entries are removed from the download directory, even the .lastdone file, and then if the program aborts because of a Chrome error, the .lastdone isn't put back for a subsequent attempt.

Didn't realize the -start flag served a similar purpose. Still, folks who have a lot of photos to process may want to look at the value in that file before restarting. Maybe running with -v also shows the last one saved.

Thanks again.