Adding in historical data

rlgbtq / TDoR2018

Datathon for the Transgender Day of Rememberance

5 stars 2 forks source link

Adding in historical data #4

Open hturner opened 5 years ago

hturner commented 5 years ago

Currently we have data from 2010.

https://transrespect.org/en/trans-murder-monitoring/tmm-resources/ (current main source) goes back to 2009; we have a Python script to extract the data.

https://docs.google.com/spreadsheets/d/10_JeZ7LEIRhvewTCoL5mVkH6MQJzf7fSXl4-P6QJ0v0/edit#gid=0 from https://tdor.info has data from 1970 to 2012.

has the data since 2009 been merged with the tgeu data?
can we write a script to extract the pre 2009 data and create monthly csvs?
has the data since 2012 on https://tdor.info been merged?

annajayne commented 5 years ago

I have the 2009 data nearly ready to upload, and I'm already working back over the historical data from tdor.info checking it and converting to the same format as the rest.

The 2010-2012 data which overlaps between tdor.info and TGEU has already been merged.

FYI https://en.wikipedia.org/wiki/List_of_unlawfully_killed_transgender_people is another source which is worth being aware of - especially as it includes links to original sources.

brennanpincardiff commented 5 years ago

Do we need something to compare data from different sources and add links to sources if available?

hturner commented 5 years ago

Sounds like scraping data since 2012 from tdor.info and data from all years from Wikipedia are on the TODO.

Then yes, we would want a way to compare data - not sure if we would automatically want to add links before checking them, maybe add but mark for review?

hturner commented 4 years ago

@annajayne I think you have added all historical data now, or is there something we can help with?