search

ArchiveTeam / ArchiveBot

ArchiveBot, an IRC bot for archiving websites
http://www.archiveteam.org/index.php?title=ArchiveBot
MIT License
356 stars 72 forks source link

Detect MediaWiki and retrieve Special:AllPages #505

Open upintheairsheep opened 3 years ago

upintheairsheep commented 3 years ago

If it detects it is a mediawiki wiki, it should go to Special:AllPages

Ex: https://apple.fandom.com/wiki/Special:AllPages

JustAnotherArchivist commented 3 years ago

In virtually all cases, Special:AllPages can be discovered from the homepage on a recursive crawl. On standard MediaWiki installations, there's a link to Special:SpecialPages in the sidebar, which in turn links to AllPages (among various other things). On heavily modified ones, like Fandom, it can be a bit less obvious, but there is a direct link to AllPages in the right sidebar on the homepage. Over the years, I haven't come across a single wiki where this addition would've been necessary. So I'm not fundamentally opposed to this, but the benefit is marginal to inexistent.