search

openzim / gutenberg

Scraper for downloading the entire ebooks repository of project Gutenberg
https://download.kiwix.org/zim/gutenberg
GNU General Public License v3.0
126 stars 37 forks source link

Download files using scraperlib #128

Closed satyamtg closed 4 years ago

satyamtg commented 4 years ago

We currently download files using cURL. https://github.com/openzim/gutenberg/blob/825470482e30bd3ef70e79e587780504c0fc7fa4/gutenbergtozim/utils.py#L101

It would be better if we have those downloaded using scraperlib as that would mitigate the risk of failures

rgaudin commented 4 years ago

Agrees.