search

santosderek / Imgur-To-Folder

Download Imgur albums and images to desired folder
Other
80 stars 9 forks source link

Downloading large favourites album #2

Closed Fynkymynky closed 1 year ago

Fynkymynky commented 6 years ago

Hi,

Thank you for this, it works really well. I do have on issue, when I run

imgurToFolder --download-all-favorites userfoo

It download the first batch of images but then stops, I have over 1,000 images in my favourites, is it possible to paginate through a large collection? thank you

santosderek commented 6 years ago

Hello!

Currently there is no way to paginate through a large collection but I will add it in. I have to look further into why there is a cap of favorites being collected.

Sorry for the inconvenience.

santosderek commented 6 years ago

After looking into ImgurPython (the module I've been using to request from Imgur), I have found that Imgur no longer supports this package as of two months ago.

I have found that the limit of favorited items that can be requested at a time are the latest 60 albums. Due to this limitation I have changed the descriptions the respective commands to reflect this 60 item limitation, for the time being.

I intend to use Imgur's restful api to overcome this obstacle, but this may take time to implement so until further updates, I will keep the descriptions to reflect the 60 item limitation.

Again, sorry for the inconvenience!

Fynkymynky commented 6 years ago

Sorry for not getting back to you sooner. Thank you very much for looking into this and you haven't caused any inconvenience :)

Thank you for the great work.

santosderek commented 6 years ago

Hello!

I have now added a new implementation to the way favorited items are downloaded. Theoretically, there shouldn't be any limit to how many favorited items are downloaded, but Imgur does limit to about 12,500 requests per day on their side as I have found out through their documentation.

Since there is now a new implementation, the way of authenticating has changed and steps can be found at the bottom of the README.md.

Please let me know if any issues pop-up with the program!

yugaanks commented 6 years ago

Hi, I have 750 images in favorites and after following the instructions in the description, I could download all the files successfully. Thank you. Sync Question (nevermind- see edit) I have a question- Is there a way to sync the folder- For example, only download the files that are not in the current folder. Say if one day I have more than 12500 images in favorites, and I run the app again, will it re-download all the files which are already in the current folder and overwrite them? or is there a way to skip those files and not count them as a request, so only download the latest files. Earlier it was mentioned that you can download the latest 60 files but I see the command is same for both, latest 60 files and download all files in favorites. If there is a way to download only the latest 50 or 60 files, I could just periodically run that command and sync it manually. I apologize if I failed to understand the description. Edit- Okay, my bad, I see the difference in download-all-favorites and latest-favorites. Anyways, Thank you for the app. Nice work.

santosderek commented 6 years ago

The program will skip over files that it has already downloaded; so if you run it a second time it will fetch a list of all your favorites and download the missing items. This is based on file name so if the file name is not found within the download folder it will download the missing item.

I plan to add more options to the downloading of favorites, including grabbing the latest "X" number of favorites, with the user specifying the "X", as well as adding an "oldest" option as well.

yugaanks commented 6 years ago

that would be a really great feature to download the latest X images only. In my case, I use google back up and sync program- downloading the images from imgur to folder to adding them to google photos, and when it's done, I remove the images from the folder. That is why downloading only the latest 50 or 60 images would be suitable for me instead of downloading all the favorites over and over.

santosderek commented 6 years ago

Hello!

I have added the ability to sort by newest and oldest favorited items, and also added the ability to limit the amount of favorites to download. 

imgurtofolder --download-latest-favorites [username] --max_favorites [maximum_number_of_favorites]

[OR]

imgurtofolder --download-oldest-favorites [username] --max_favorites [maximum_number_of_favorites]

These commands will download your latest / oldest favorited items up to the number specified. You could also not include --max_favorites [maximum_number_of_favorites] and download all your favorites in order from latest / oldest. 

imgurtofolder --download-latest-favorites [username] 

[OR]

imgurtofolder --download-oldest-favorites [username]

These two commands will hopefully offer better ease in your backing up and syncing to google drive.

Lastly, I have added --overwrite incase pictures needed to be overwritten.

If there is any complications / improvements you have, feel free to ask!

yugaanks commented 6 years ago

It works! Thank you. exactly what I needed.

yugaanks commented 6 years ago

hey, in description, it says imgurToFolder --download-latest-favorites [username] --max_favorites [maximum_number_of_favorites]
but command works with --max-favorites

santosderek commented 6 years ago

Hello!

Thank you for letting me know!

I just fixed --max-favorites to alert user if no number was given. Before it was acting as if --max-favorites was not even called and downloaded all favorites.

I also added a better example to the README and changed the headers.

Just let me know if there was anything else that I could improve!