added scraper for terrorism incidents, and the output of the links an…

Data4Democracy / immigration-connect

Building tools to connect and coordinate efforts to help those affected by immigration law changes in partnership with the NILC

42 stars 29 forks source link

added scraper for terrorism incidents, and the output of the links an… #24

Closed rhoneybul closed 7 years ago

rhoneybul commented 7 years ago

I have added a scraper, which scrapes wikipedia events from 1970-onwards and outputs the results in a json file

I used json instead of csv because the pages were not exactly the same, so the headers did not remain consistent from page to page

rhoneybul commented 7 years ago

I have made requested changes, mainly increasing the amount of flexibility with regards to specifying outputs. I have included a fairly basic data schema based on the 1994 data.

jtorrez commented 7 years ago

Things have been crazy at work lately and I haven't had a chance to get to this, will review in next week. Glancing over it changes look good

jtorrez commented 7 years ago

LGTM, merging this. Thanks for your patience and pinging me again @rhoneybul