search

Data4Democracy / immigration-connect

Building tools to connect and coordinate efforts to help those affected by immigration law changes in partnership with the NILC
42 stars 29 forks source link

#28 - scraped tweets on key legislators #38

Closed joshuAnalytics closed 6 years ago

joshuAnalytics commented 7 years ago

Tweets were collected on key legislators @rongant,@senatornorris,@billketron,@jimtracy over the date range --since 2012-01-01 --until 2017-10-28.

I used Get Old Tweets, a python script which scrapes the twitter website to retrieve old tweets which are not available via the public search API. I amended the Exporter.py script, replacing the ';' separator with '\t' to handle tweets with semicolons. This generated a tab separated values file which I read with python pandas, and wrote out to json as this was the format of the other data files in the datasets directory. I updated the readme with info on the dataset.

jtorrez commented 7 years ago

@joshuAnalytics, sorry totally didn't see this til now! Could you please update the PR to follow the contribution guidelines. Less concerned about you needing to re-branch, but referencing the issue in this PR and a general description of the approach you took so we can be sure that the issue should be closed would be great.

joshuAnalytics commented 7 years ago

@jtorrez , apologies, this was my first ever git push! Have amended the PR name and added a description, let me know if I've done this right.

joshuAnalytics commented 6 years ago

@jtorrez did you get a chance to review?