search

Data4Democracy / usa-dashboard

A dashboard of key metrics for the USA
69 stars 27 forks source link

San Diego Crime data #39

Open srcole opened 7 years ago

srcole commented 7 years ago

SANDAG (San Diego Association of Governments) releases the past 6 months of crime data (in .csv format!) every Wednesday.

The data can be accessed at this link: http://www.sandag.org/programs/public_safety/arjis/CrimeData/crimedata.zip

By looking at the other city folders, it looks like the desired format is:

  1. a .py file for the scraping and the .csv file with the columns year; month; day; offense; count.
  2. the .csv

I'm happy to get the data in that format in the next week or so.

Since they only release 180 days at a time, the first scraper will get all 180 days, and subsequently it can be run once per week to get the next 7 days.

seanjtaylor commented 7 years ago

@srcole That would be great! Is there any chance of getting historical data?

srcole commented 7 years ago

@seanjtaylor Cool!

From SANDAG, unfortunately not, the FAQ on a related mapping app said that you'd have to contact the local authorities for anything older than 180 days. I'll keep my eyes out though.

There is this table that has monthly totals of crimes since 2008. But that doesn't seem to fit with the day-to-day format we currently have. Unfortunately I can't find anything better right now.

Even without the historical data, the scraper of the last 180 days (and future) will still be useful, right?