search

Data4Democracy / usa-dashboard

A dashboard of key metrics for the USA
69 stars 27 forks source link

Scrape Crime Data for Dallas #51

Open bobpore opened 7 years ago

bobpore commented 7 years ago

Looks like Dallas has a pretty good and up to date API with documentation for their crime data here. I will produce a scraper for the data.

bobpore commented 7 years ago

After taking some time to play around with the API and look at the data, I'm not positive this is going to be a great source.

  1. The historical data is very limited. The data set is described as beginning June 1, 2014, but I find that questionable. There are a few dozen rows with incidents prior to June 1, 2014, but the bigger problem is that the amount of data from June 1st through the end of 2014 seems way to low. There are only 235 rows for the last 7 months of 2014 whereas 2015 and 2016 have 57K rows and 58K rows respectively for the same time period.

  2. The data seems to be incomplete even for 2015 and beyond where the number of rows increases substantially. According to the database, there were 8 Murders in January 2017. According to this police report, there were 17. Similar inconsistencies exist for other crime data in the API versus police and media reports I've found. I think the cause of the discrepancy is the privacy policy the Dallas Police Department employs. The API documentation reads:

This online site is an attempt to make it easier for citizens to access offense reports. In disseminating this crime information, we must also comply with current laws that regulate the release of potentially sensitive and confidential information. To ensure that privacy concerns are protected and legal standards are met, report data is "filtered" prior to being made available to the public. Among the exclusions are:

1.) Sexually oriented offenses 2.) Offenses where juveniles or children (individuals under 17 years of age) are the victim or suspect 3.) Listing of property items that are considered evidence 4.) Social Service Referral offenses 5.) Identifying vehicle information in certain offenses

Additionally, the Dallas Police department's website reads: `

This website represents a filtered version of the data contained within that application for the purpose of the sharing of non-sensitive data with the citizens of Dallas. This data should not be used for statistical purposes as it only represents data that is releasable to the General Public.

`

So... dang. I haven't been able to find another source for Dallas crime data.