openoakland / OakCrime-Decommissioned

Code supporting citizen analysis of crime in Oakland, CA
22 stars 16 forks source link

Pre-compute geographic columns to all incidents #12

Closed rbelew closed 3 years ago

rbelew commented 7 years ago

Associating the crime incidents with various geographic regions can be computationally expensive, especially in trying to satisfy queries quickly. Several types of regions are going to be especially useful towards integrating crime data with other data sources:

For every kind of data, the process involves:

the views.add_zip() function shows how to do this. (but note that this code does NOT include the update efficiency, it uses an incremental save()).

One issue is that the currently provisioned OakCrime server is not set up to allow long-term, memory-intensive processes like these, and so the new geo feature needs to be computed elsewhere and then added as a column (via postgres update).