everypolitician / democratic-commons-tasks

An issues-only repository for tracking work for the Democratic Commons project
0 stars 0 forks source link

create build tools for the repo-per-country Democratic Commons data #39

Open mhl opened 6 years ago

mhl commented 6 years ago

[I think @tmtmtmtm and @chrismytton would be in a better position to write this ticket, so please assume that this pretty much a placeholder, or an opportunity to refine our collective understanding of what's required.]

The role of these tools is to take data from Wikidata about politicians in a particular country and generate easily usable data: CSV files and Popolo JSON or Popolo JSON-LD.

These tools should in some way allow us to do sanity checks on the changes in data coming from Wikidata; this might be via a similar mechanism as for everypolitician-data (where a new version of the data is generated, and a pull request is made with that which someone has to review) or there could be some other interface to let us or partners check the incoming changes and fix any problems.

(This should be easier than with everypolitician-data pull requests since there is only a single source of data (Wikidata) not multiple sources that are being merged.)

To set up a new country repository, I'd anticipate the process being something like:

... which would then build the CSV and JSON.

Note that:

mhl commented 6 years ago

A start on such tools (but in-repository for the moment) for Canada is here: https://github.com/everypolitician/proto-commons-canada/ - note also that this only builds a single legislature at the moment.