search

DMarby / humblebundle-ebook-downloader

An easy way to download ebooks from your humblebundle account
The Unlicense
230 stars 27 forks source link

Add State File for Comparing "already exists" #52

Open markjx opened 3 years ago

markjx commented 3 years ago

First; I'm a huge fan of the project. THANKSf!

I like being able to use the "--format all --all" flags to sync my local folder after getting a few new bundles. But, my library is getting pretty big and the time to go through all the files and figure out that the download "already exists" takes significant time and seems to put undue load on the humblebundle.com servers.

Two ideas:

  1. When there's a directory for the name of a previously downloaded bundle, then assume that all files in that bundle are already downloaded. Then don't check each file. Maybe a "--skip-previous-bundles" flag to enable this different functionality?
  2. Keep a local state file containing info about previously downloaded files. Then, just get the new ones. I'm thinking of something similar to the "--download-archive " flag in youtube-dl

My library has > 200 bundles and >8000 files. The humblebundle-ebook-downloader seems to be less reliable as my library is getting bigger. My most recent run crashed after 18 minutes only making it through <4000 / 8000 files. I feel like scoping it down may make the software more reliable.

Tokariew commented 2 years ago

i will plug myself a little… https://github.com/Tokariew/hb_downloader maybe not to late ;)