search

WikiTeam / wikiteam

Tools for downloading and preserving wikis. We archive wikis, from Wikipedia to tiniest wikis. As of 2024, WikiTeam has preserved more than 600,000 wikis.
https://github.com/WikiTeam
GNU General Public License v3.0
730 stars 151 forks source link

New wikiteam bot for wikiapiary.com #457

Open yzqzss opened 1 year ago

yzqzss commented 1 year ago

Who is controlling the wikiteam bot on wikiapiary.com? Is it possible to deploy this new bot?

https://github.com/saveweb/wikiapiary-wikiteam-bot

(This repo can go into wikiteam/wikiteam.)

WikiTeam bot has been inactive since 2016, I got it working again and a little more robust :-D

yzqzss commented 1 year ago

@emijrp

yzqzss commented 1 year ago

I deployed it (Savewebbot) and It works.

Now it always pushes the latest dump's information to wikiapiary and can update existing IA parameters in wikitext to the latest dump. (adapted the --append-date parameter of uploader.py)

I also use sqlite to store the "last successfully checked date" and IA-identifier for each wiki, so the bot can adjust the update interval and skip them, speeding up the secondary traversal.

nemobis commented 1 year ago

Thanks for running the bot! There's no need to use the old account.

If you want we can also move the repository under the WikiTeam org. I'll then ask emijrp to add you to the WikiTeam group.