search

mediawiki-client-tools / mediawiki-dump-generator

Python 3 tools for downloading and preserving wikis
https://github.com/mediawiki-client-tools/mediawiki-scraper
GNU General Public License v3.0
95 stars 14 forks source link

Update getPageTitlesAPI in generator.py #4

Closed t-karcher closed 2 years ago

t-karcher commented 2 years ago

Add delay after retrieving the titles from every single namespace: The wiki I try to backup obviously has very strict API rate limits, which lead to refused connections if the delay is not added between every single request.

elsiehupp commented 2 years ago

Hi @t-karcher—I will have to get to this after I merge @vgambier's PR, since I had a bunch of unpushed changes that made merging overly complicated. If you want to help, you can pop over there and test the merged version yourself (since I don't have Windows at the moment).

elsiehupp commented 2 years ago

All ready for you to rebase here!

elsiehupp commented 2 years ago

I mean, I can probably rebase this myself.