openzim / gutenberg

Scraper for downloading the entire ebooks repository of project Gutenberg
https://download.kiwix.org/zim/gutenberg
GNU General Public License v3.0
127 stars 37 forks source link

Make one zim for each language ? #55

Closed dattaz closed 6 years ago

dattaz commented 6 years ago

Currently we have book in each of these languages (nb is number of book in this language) :

{'lang': u'ko', 'nb': 1}
{'lang': u'sa', 'nb': 1}
{'lang': u'ang', 'nb': 1}
{'lang': u'yi', 'nb': 1}
{'lang': u'lt', 'nb': 1}
{'lang': u'myn', 'nb': 1}
{'lang': u'ia', 'nb': 1}
{'lang': u'kld', 'nb': 1}
{'lang': u'oc', 'nb': 1}
{'lang': u'nap', 'nb': 1}
{'lang': u'gl', 'nb': 1}
{'lang': u'nah', 'nb': 1}
{'lang': u'nav', 'nb': 1}
{'lang': u'br', 'nb': 1}
{'lang': u'iu', 'nb': 1}
{'lang': u'bgs', 'nb': 1}
{'lang': u'rmr', 'nb': 1}
{'lang': u'sl', 'nb': 1}
{'lang': u'oji', 'nb': 1}
{'lang': u'ar', 'nb': 1}
{'lang': u'et', 'nb': 1}
{'lang': u'fa', 'nb': 1}
{'lang': u'brx', 'nb': 1}
{'lang': u'ro', 'nb': 2}
{'lang': u'ga', 'nb': 2}
{'lang': u'fy', 'nb': 2}
{'lang': u'mi', 'nb': 2}
{'lang': u'arp', 'nb': 2}
{'lang': u'gla', 'nb': 2}
{'lang': u'grc', 'nb': 2}
{'lang': u'nai', 'nb': 3}
{'lang': u'ilo', 'nb': 3}
{'lang': u'ceb', 'nb': 3}
{'lang': u'sr', 'nb': 4}
{'lang': u'af', 'nb': 4}
{'lang': u'enm', 'nb': 5}
{'lang': u'bg', 'nb': 6}
{'lang': u'he', 'nb': 6}
{'lang': u'fur', 'nb': 6}
{'lang': u'te', 'nb': 6}
{'lang': u'is', 'nb': 7}
{'lang': u'ru', 'nb': 9}
{'lang': u'cy', 'nb': 10}
{'lang': u'cs', 'nb': 10}
{'lang': u'no', 'nb': 19}
{'lang': u'ja', 'nb': 22}
{'lang': u'pl', 'nb': 31}
{'lang': u'ca', 'nb': 32}
{'lang': u'tl', 'nb': 57}
{'lang': u'da', 'nb': 68}
{'lang': u'la', 'nb': 99}
{'lang': u'eo', 'nb': 107}
{'lang': u'hu', 'nb': 175}
{'lang': u'sv', 'nb': 181}
{'lang': u'el', 'nb': 217}
{'lang': u'zh', 'nb': 436}
{'lang': u'pt', 'nb': 548}
{'lang': u'es', 'nb': 582}
{'lang': u'it', 'nb': 721}
{'lang': u'nl', 'nb': 780}
{'lang': u'de', 'nb': 1660}
{'lang': u'fi', 'nb': 1886}
{'lang': u'fr', 'nb': 2858}
{'lang': u'en', 'nb': 46641}
kelson42 commented 6 years ago

Definitly!

dattaz commented 6 years ago

https://github.com/openzim/gutenberg/commit/586c86f56d30bdfc37b09c72ff3b8f36bf90db3d add option --one-language-one-zim which make one zim per language and a global zim. If no language are specify (with -l) we search in db to find languages.