search

JeroenDeDauw / QueryrAPI

🖹 REST API for Wikibase data
https://wikibase.consulting
7 stars 0 forks source link

Add support for following redirects #23

Open addshore opened 8 years ago

addshore commented 8 years ago

Currently this thing has no idea / concept of redirects. They are important and should be accounted for.

JeroenDeDauw commented 8 years ago

Good point. At present the EntityStore component does not support redirects, and neither does the importing code (Replicator and JsonDumpReader). Actually not sure how redirects are represented in the dump, if at all.

addshore commented 8 years ago

They are not represented in the dump :/ Thought there is a bug for that https://phabricator.wikimedia.org/T98320

JeroenDeDauw commented 8 years ago

What about the API? I know it follows redirects by default, but will it somehow tell you it did, so you can also store the redirect info?

And is there a sane way to fetch all redirect info, to mitigate it not being in the dump?

addshore commented 8 years ago

https://www.wikidata.org/w/api.php?action=query&list=allredirects&arnamespace=0&arprop=ids|title&arlimit=100

Just ignore non items

Also youll have to do a second query to find the target from the pageid

JeroenDeDauw commented 8 years ago

If I'm not mistaken, some smug created 500k redirects or something, so doing this via the API will take some time :)

addshore commented 8 years ago

Hah, yes indeed, some smug did. done.txt redirects.txt