Closed wolfgang42 closed 2 years ago
@wolfgang42 Your detailed description is right and you have understood things properly. I suspect this recent ticket talking about the very same ZIM file might help you https://github.com/openzim/mwoffliner/issues/1531. Let me know.
Yes, that does look like a similar question. I see you wrote:
This zim file is based on http://download.openzim.org/wp1/enwiki/projects/Computing.tsv which is the list of articles of the computing project https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Computing.
which is about what I figured (I wasn't sure if there were other sources as well), but it's not clear to me how to get from one to the other. Is there a certain category hierarchy that gets crawled by something to produce the full article list?
At a guess, it seems like the criteria is maybe that the article's talk page is in Category:All Computing articles, probably by Template:WikiProject Computing or a derivative—is that correct, and if so where would I find the code that does that?
The wikiproject articles are gathered (to the wp1 database) by the wp1 bot (openzim/wp1) and put in TSV list by the code in this repo.
If you want to have an insight in the our wp1 database, you can go to https://wp1.openzim.org.
... and your last sentence is correct.
(Filing this here as it's the closest I've found to an upstream but please redirect me if I've ended up in the wrong place.)
I've downloaded
wikipedia_en_computer_nopic.zim
from the Kiwix wiki and am playing around with it, but I'm pretty confused by the article selection. For example, the article The Art Life (“a blog about the art scene in Sydney, Australia”) is included, but ISO 8601 (an extremely common date format) is not.I stumbled upon the zim-requests repo which lead me to the zimfarm config for this bundle, which seems to derive from a
Computing.tsv
. From what I gather via this comment I found, it seems that this list is managed by something in this (wp1_selection_tools
) repo, but for me the trail ran cold here—I poked around in the scripts briefly and didn't see anything obvious, though I didn't delve too deeply.Can you point me in the right direction to find where these lists are derived from? I assume there are categories or WikiProjects somewhere but I can't seem to find any documentation on this.