Closed andylolz closed 4 years ago
I’ve marked this as "in progress" because I’ve already created scrapers for several of the codelists in question.
I am guessing that once we have the scrapers outputting to a repo, we can have a job/tool that calls csvdiff, xmldiff etc to monitor when the scraped and the IATI version differ - and alert us - bonus points if this was output to a file that we could suck into a one page traffic light dashboard - even more bonus points if it was automatically tweeted ala IATI canary but for codelists.
A few ideas:
Ace – great stuff both.
I’ve made a new repo here: https://github.com/codeforIATI/codelists
I’ll transfer those ideas across to issues in that repo now shortly.
@markbrough has done some more on this one :tada:
I’ve moved suggestions to https://github.com/codeforIATI/codelists/issues.
Please do keep adding to that list! It would be good to get a list of features from which to assemble a bit of a spec.
This exists! https://codelists.codeforiati.org
There’s more work to do on this, but closing it here since we have a dedicated issues board for it.
Rationale
The IATI standard includes a number of replicated codelists, that come from external sources (e.g. DAC; IANA; ISO). These are currently maintained by the secretariat, but the update process is entirely manual, so there’s a lot of overhead. As a result, it’s sometimes difficult for codelists to be kept up-to-date.
Proposal
Maintain a set of these replicated codelists in IATI format. Auto-update them using scrapers, in much the same way as the EveryPolitician bot. There are currently 22 replicated codelists, so this is a very manageable number. However, I know there is interest from @matmaxgeds in expanding this to other codelists that are useful to publishers but are not currently part of the IATI standard.
The replicated codelists could be outputted in XML, JSON, CSV and nicely formatted HTML.
The name Tyrell is a reference to the replicant manufacturing “Tyrell Corporation”, in the 1982 neo-noir sci-fi film Blade Runner.We think this is probably a bit sinister! Need a better name :)