Add website classification

octopusinvitro commented 8 years ago

This PR allows website user names (like those for services like Facebook, Flickr, YouTube, etc.) that are read from a person's Wikidata page to be stored in the Popolo file as identifiers.

In our way there, we also refactored some smelly code and simplified complex methods

Notes to reviewer:

A new VCR cassette was loaded for Konstantinos Mitsotakis, since he has two website user names: YouTube and Flickr.
The P553 id in Wikidata refers to websites, which is the field from where we are reading the website user names. We are using it in the code to fetch the websites information.
The website user names have an id of P554 that is mandatory (will always be present if P553 is present), so we are hardcoding it safely but adding a rescue just in case.

Closes https://github.com/everypolitician/everypolitician/issues/519

octopusinvitro commented 8 years ago

@tmtmtmtm what's the error, could you copy paste it please. Sounds like it could potentially be an easy fix

tmtmtmtm commented 8 years ago

sqlite3-1.3.11/lib/sqlite3/database.rb:91:in `initialize': near "world": syntax error (SQLite3::SQLException)

(you can test this by running that scraper yourself locally with the Gemfile changed to point at this branch)

tmtmtmtm commented 7 years ago

This version failed quite nastily when the site name was something like My World@Mail.Ru. Superseded by https://github.com/everypolitician/wikidata-fetcher/pull/51

everypolitician / wikidata-fetcher

Add website classification #31

Notes to reviewer: