Core datasets complex scripts bugs

[Branko-Dj]

https://github.com/datahq/pm/issues/292

As a PM I want to fix the scripts for core datasets that have bugs which have proven more troublesome to fix

Analysis

1. cofog

Source for process.py script on line 20 access_db_zip_url = 'http://unstats.un.org/unsd/cr/registry/regdntransfer.asp?f=186' is broken. Moving it to broken sources

2. corruption-perceptions-index

Loads of problems. Libraries need to be installed, sometimes source is downloaded but it is not unzipped, some sources unavailable, it's a mess. I suggest complete refactoring of this script

3. gdp-us A slight change in url for sources was the problem, easy fix. It might be useful refactoring script to use python 3

4. geo-admin1-us This uses tuttle file so further analysis is needed

5. geo-countries This uses tuttle file so further analysis is needed

6. geo-ne-admin1 This uses tuttle file so further analysis is needed

7. house-prices-global Format of the source csv file was changed so the script was slightly modified in order to work

8. house-prices-uk Script is workin properly, the problem was in instructions for running the script in README which is now fixed

9. imf-weo Unable to make script work, some problem with locales

10. interest-rates-gb Script works properly but the source is outdated and the script should be refactored

11. language-codes Unable to make PHP work, further analysis needed

12. pharmaceutical-drug-spending Fixed importing libraries in population.py

13. s-and-p-500 In progress as a separate issue

14. s-and-p-500-companies In progress as a separate issue

15. un-locode Unable to make PHP work, further analysis needed

16. world-cities This uses tuttle file so further analysis is needed

Acceptance criteria

• [ ] The scripts with complex bugs are working properly. Those that require further analysis are labeled as such.

Tasks

• [x] Determine and fix the bugs (if possible) or move repo to the [broken source issue(https://github.com/datahq/pm/issues/291) for the following list of datasets
  • [x] https://github.com/datasets/cofog
  • [x] https://github.com/datasets/corruption-perceptions-index
  • [x] https://github.com/datasets/gdp-us
  • [x] https://github.com/datasets/geo-admin1-us
  • [x] https://github.com/datasets/geo-countries
  • [x] https://github.com/datasets/geo-ne-admin1
  • [x] https://github.com/datasets/house-prices-global
  • [x] https://github.com/datasets/house-prices-uk
  • [x] https://github.com/datasets/imf-weo
  • [x] https://github.com/datasets/interest-rates-gb
  • [x] https://github.com/datasets/language-codes
  • [x] https://github.com/datasets/pharmaceutical-drug-spending
  • [x] https://github.com/datasets/s-and-p-500
  • [x] https://github.com/datasets/s-and-p-500-companies
  • [x] https://github.com/datasets/un-locode
  • [x] https://github.com/datasets/world-cities

[Branko-Dj]

Some datasets are fixed, but there are a few complex ones that use PHP and tuttle that will require further analysis and are a part of a separate issue