search

WikiTeam / wikiteam

Tools for downloading and preserving wikis. We archive wikis, from Wikipedia to tiniest wikis. As of 2024, WikiTeam has preserved more than 600,000 wikis.
https://github.com/WikiTeam
GNU General Public License v3.0
714 stars 148 forks source link

Use API (tracking) #44

Open emijrp opened 10 years ago

emijrp commented 10 years ago

From nemow...@gmail.com on March 01, 2012 00:41:27

Many bugs request API usage or could (probably) be fixed by using API. Using API for pages is already many many times more efficient than index; using API everywhere (on wikis which have them) would make the script more robust and faster.

Original issue: http://code.google.com/p/wikiteam/issues/detail?id=44

emijrp commented 10 years ago

From ad...@alphacorp.tk on June 22, 2012 03:02:50

Summary: Use API (tracking)
Labels: Performance
Blockedon: -wikiteam:10 -wikiteam:19 -wikiteam:21 -wikiteam:22 -wikiteam:24 -wikiteam:32 -wikiteam:37 wikiteam:10 wikiteam:19 wikiteam:21 wikiteam:22 wikiteam:24 wikiteam:32 wikiteam:37

emijrp commented 10 years ago

From nemow...@gmail.com on November 09, 2012 02:05:50

Blockedon: wikiteam:26 wikiteam:56

emijrp commented 10 years ago

From nemow...@gmail.com on November 09, 2012 02:15:26

There are some libraries which might help: https://www.mediawiki.org/wiki/API:Client_code#Python

emijrp commented 10 years ago

From nemow...@gmail.com on May 04, 2013 03:18:00

See also specifications for a rewrite at https://meta.wikimedia.org/wiki/WikiTeam/Dumpgenerator_rewrite

emijrp commented 10 years ago

From nemow...@gmail.com on October 25, 2013 05:46:35

Cc: phoenixo...@gmail.com

emijrp commented 10 years ago

From nemow...@gmail.com on November 05, 2013 09:48:39

Blocking: wikiteam:49

emijrp commented 10 years ago

From ad...@alphacorp.tk on November 11, 2013 22:38:57

Blocking: wikiteam:75

emijrp commented 10 years ago

From nemow...@gmail.com on December 16, 2013 04:02:07

Interesting set of PHP scripts which do some things more than dumpgenerator.py for API-enabled wikis (like logs download; interwiki map and usergroups are rarely interesting). https://www.mediawiki.org/wiki/Manual:Grabbers

emijrp commented 10 years ago

From nemow...@gmail.com on June 08, 2014 06:12:35

As for XML download:

jayvdb> Nemo_bis: there is an export=1 and exportnowrap=1 API params, which force the API to provide the same as Special:Export, right down to the bugs in Special:Export

Also, his https://gerrit.wikimedia.org/r/136352 should remove any obstacle in using pywikibot.