Vultaire / shutterrip

A Shutterfly Share Sites Download Tool
GNU General Public License v3.0
6 stars 0 forks source link

Pulling individual albums only. (This has stopped 25 of 284 albums listed in sharesite) #1

Open sburgan opened 1 year ago

sburgan commented 1 year ago

This has worked great for me with the first 25 albums, (out of 280) but it gets to a point where it skips the site listed and won't find anything else (it assumes it has completed). I attemtped to put in individual album links in the config file, and although it will open them, it does not proceed to download anything and times out and just repeats the process on the same link.

Is this from a code change from shutterfly? I tried reading the html in my root sharesite, but it appears this comes from javascript. Were you able to pull individual albums, without the code iterating from the root link? (ie, just input an album link) and let it pull just from that album.

I cant tell what/where it is determining that the whole site has been downloaded.

Vultaire commented 1 year ago

Hi,

Glad someone else is giving this a try. And no, I don't think they've changed anything on the Shutterfly site; it's likely a subtle difference in what we're doing here.

As you probably already saw, I'm not really supporting this - but I'll tell you the process I used for prepping the share sites list. I'll go step by step here, probably repeating stuff that is obvious to you, but maybe we can find some small difference between our processes and that might help you out here.

Here's the main process I sued:

  1. Make sure you're logged into shutterfly.com.
  2. Open the front page of the share site in question. (E.g. myalbums2022.shutterfly.com)
  3. Click the "Pictures & Videos" tab at the top. (The URL will then read e.g. myalbums2022.shutterfly.com/pictures.)
  4. Once that page loads, just under the navigation bar you'll see the header "Pictures". Click that. (Or alternatively, if there is a link that says e.g. "View all 21 albums" near the bottom of the page, that'll work as well.)
  5. Copy the link of this page. From my experience, it's always been something like myalbums2022.shutterfly.com/pictures/8 - that is, specifically with the /8 suffix. I don't know how consistent that is, but for me it has been.

Now, for one of the albums I had to back up, there were pictures stored outside of the Pictures & Videos section. I'm not a Shutterfly expert, but rather just a dad who can code that wanted to back up preschool pictures of his kids, so I don't really fully understand how all this works, but anyway...

For that one other album I had to back up, the process was like this:

  1. Make sure you're logged into shutterfly.com.
  2. Open the front page of the share site in question (e.g. myalbums2022.shutterfly.com)
  3. From this home page, you may see some other categories of pictures shown directly here. (In my case it was "写真", Japanese for pictures.) Click on the category name.
  4. Copy the URL of the page which opens. In this case, since it's outside the normal "Pictures & Videos" section, it's something like myalbums2022.shutterfly.com/70, i.e. without the "/pictures" subfolder.

Ultimately, what it comes down to is this: you want to copy the URL of the page where you can see all the albums. If you can get to a album listings page, and then click on the "All" link to display all your albums - that's the URL you want to copy. The script just pulls all the albums from that page (automatically clicking the "All" link), then spiders through each of the albums in turn.

I hope this helps you out! Best of luck!

Vultaire commented 1 year ago

Hi,

A quick follow-up. I re-read your issue, and saw:

I cant tell what/where it is determining that the whole site has been downloaded.

There's a very simple state file, ".state", which is created by the script in the current working directory. It's simply a Python dictionary with URLs as keys and true as the value. If you really want to re-process a given page, you can remove its URL from the state dictionary.

This being said, I suspect this is more of a process issue (see my previous comment) rather than a "I pulled this site and it didn't pull all my stuff" type of issue. But I could be wrong!

To be extra explicit, below is the code which controls marking and checking whether things are downloaded. It's very straightforward.

https://github.com/Vultaire/shutterrip/blob/920ea928306b0f628aec02611dd707579dbbc26c/shutterrip.py#L198-L218

Anyway - I wish you luck! Hope something I wrote here might help, and in the worst case, you have a starting point from which you can start hacking things if you feel so inclined.

sburgan commented 1 year ago

Hi Paul!

First: Thanks for taking the time to respond to this at all, I appreciate it..

Yes, I have been pouring over the code and determined where things were, but (since I am not a programmer by day, i'm just learning as I go along.) I am going to goof around in it again tonight, its just weird how it stops at this one area of albums and won't proceed any further. I am trying to step through and see where its hanging up and its as if its not able to determine an album link, is an actual album (per the code).

Let me see what happens if i comment some checks off and what it does..

Thanks again and i will be in touch (just to update you what I find..)

thanks paul!

On Mon, Feb 20, 2023 at 11:45 PM Paul Goins @.***> wrote:

Hi,

A quick follow-up. I re-read your issue, and saw:

I cant tell what/where it is determining that the whole site has been downloaded.

There's a very simple state file, ".state", which is created by the script in the current working directory. It's simply a Python dictionary with URLs as keys and true as the value. If you really want to re-process a given page, you can remove its URL from the state dictionary.

This being said, I suspect this is more of a process issue (see my previous comment) rather than a "I pulled this site and it didn't pull all my stuff" type of issue. But I could be wrong!

To be extra explicit, below is the code which controls marking and checking whether things are downloaded. It's very straightforward.

https://github.com/Vultaire/shutterrip/blob/920ea928306b0f628aec02611dd707579dbbc26c/shutterrip.py#L198-L218

Anyway - I wish you luck! Hope something I wrote here might help, and in the worst case, you have a starting point from which you can start hacking things if you feel so inclined.

— Reply to this email directly, view it on GitHub https://github.com/Vultaire/shutterrip/issues/1#issuecomment-1437892546, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKT6WPQLMY5RVTIPZPFVALDWYRJAFANCNFSM6AAAAAAVBCUJBY . You are receiving this because you authored the thread.Message ID: @.***>