Tumblr Download Tool with High Speed and Customization.
$ git clone git@github.com:tzw0745/tumblr-crawler-cli.git
$ cd tumblr-crawler-cli
$ pip install -r requirements.txt # -i https://pypi.tuna.tsinghua.edu.cn/simple/
$ python tumblr-crawler.py --help
NOTICE: if you want use socks proxy for this program, you need install another package: pySocks
$ pip install pySocks
usage: tumblr-crawler.py [-h] [-p] [-v] [-d SAVE_DIR] [-f FN_FMT] [-x PROXY]
[-n THREAD_NUM] [--min MIN_SIZE] [--overwrite]
[--interval INTERVAL] [--retries RETRIES]
sites [sites ...]
Crawler Tumblr Photos and Videos
positional arguments:
sites tumblr sites
optional arguments:
-h, --help show this help message and exit
-p, --photo whether to download photo
-v, --video whether to download video
-d SAVE_DIR, --dir SAVE_DIR
download file save directory
-f FN_FMT, --format FN_FMT
filename format
-x PROXY, --proxy PROXY
http request agent, support http/socks
-n THREAD_NUM, --thread THREAD_NUM
number of download threads, default is 5
--min MIN_SIZE minimum size of downloaded files, default is 0k
(unlimited)
--overwrite overwrite file (if it exists)
--interval INTERVAL http request interval, default is 0.5 (seconds)
--retries RETRIES http request retries, default is 3
you want download all photos and videos from tumblr @liamtbyrne & @lizclimo:
$ python tumblr-crawler.py liamtbyrne lizclimo
specify the download file type:
$ python tumblr-crawler.py -p liamtbyrne # download photos only
$ python tumblr-crawler.py --video liamtbyrne # download videos only
you want put download all files to another directory:
$ python tumblr-crawler.py -d /somedir/ liamtbyrne
you want custom filename format:
$ python tumblr-crawler.py -f "{date:%Y-%m-%d %H.%M.%S} GMT.{post_id}.{uid}" liamtbyrne # default
$ python tumblr-crawler.py --format {uid} liamtbyrne
first example will save file like: "2015-10-16 06.04.53 GMT.13126.5pzVb1s7wpcjo10.jpg" second example will save file like: "5pzVb1s7wpcjo10.jpg"
{uid}
is necessary, other optional parameter include:
{post_id}
: id of tumblr post, like13126
;{type}
:video
orphoto
;{date}
: datetime of tumblr post, support detailed settings;{timestamp}
: unix timestamp, like1541405838
.
you want use proxy for download files:
$ python tumblr-crawler.py --proxy http://127.0.0.1:1080 liamtbyrne # http proxy
$ python tumblr-crawler.py -x socks5h://127.0.0.1:1080 liamtbyrne # socket5 proxy
you want set more thread to speed up the download speed:
$ python tumblr-crawler.py -n 20 liamtbyrne
you only want to download files larger than a certain size:
$ python tumblr-crawler.py --min 0.5m liamtbyrne # only download files larger than 512k
$ python tumblr-crawler.py --min 100k liamtbyrne # only download files larger than 100k