This code is going to be used to scrape the UCPD Daily Incident page at a pre-determined frequency and store the incidents on a generic JSON data-store.
Added a GoogleMaps fallback for addresses that can't be found in the Census API.
Checklist before requesting a review
[x] The code runs successfully.
make twenty_days output
```terminal
(ucpd-incident-scraper-py3.11) michaelp@MacBook-Air-5 ucpd-incident-scraper % make twenty_days
python -m incident_scraper days-back 20
Beginning the UCPD Incident scraping process.
Finished with the UCPD Incident scraping process.
97 total incidents were scraped from the UCPD Incidents' site.
API queries_quota: 60
This incident has a malformed date: {'Incident': 'Information / Burglary', 'Location': 'Kimbark between 56th & 57th', 'Reported': '8/29/223 10:36 AM', 'Occurred': '8/28/23 to 8/29/23 6:00 PM to 8:45 AM', 'Comments / Nature of Fire': 'Two bicycles taken from off-campus residential garage / Referred to CPD', 'Disposition': 'Referred', 'UCPD_ID': '2023-030093'}
1 of 30 contained malformed or voided information.
0 of 30 could not be processed by the Census or GoogleMaps' Geocoder.
29 of 30 incidents were successfully processed.
Adding 29 of 30 incidents to the GCP Datastore.
Completed adding 29 of 30 incidents to the GCP Datastore.
This incident contains voided information: {'Incident': 'Void', 'Location': 'Void', 'Reported': 'Void', 'Occurred': 'Void', 'Comments / Nature of Fire': 'Number generated in error', 'Disposition': 'Void'}
This incident contains voided information: {'Incident': 'Void', 'Location': 'Void', 'Reported': 'Void', 'Occurred': 'Void', 'Comments / Nature of Fire': 'Number generated in error', 'Disposition': 'Void'}
This incident contains voided information: {'Incident': 'Void', 'Location': 'Void', 'Reported': 'Void', 'Occurred': 'Void', 'Comments / Nature of Fire': 'Number generated in error', 'Disposition': 'Void'}
3 of 30 contained malformed or voided information.
0 of 30 could not be processed by the Census or GoogleMaps' Geocoder.
27 of 30 incidents were successfully processed.
Adding 27 of 30 incidents to the GCP Datastore.
Completed adding 27 of 30 incidents to the GCP Datastore.
0 of 30 contained malformed or voided information.
0 of 30 could not be processed by the Census or GoogleMaps' Geocoder.
30 of 30 incidents were successfully processed.
Adding 30 of 30 incidents to the GCP Datastore.
Completed adding 30 of 30 incidents to the GCP Datastore.
0 of 7 contained malformed or voided information.
0 of 7 could not be processed by the Census or GoogleMaps' Geocoder.
7 of 7 incidents were successfully processed.
Adding 7 of 7 incidents to the GCP Datastore.
Completed adding 7 of 7 incidents to the GCP Datastore.
4 of 97 incidents were NOT added to the GCP Datastore.
Program shutting down, attempting to send 5 queued log entries to Cloud Logging...
Waiting up to 5 seconds.
Sent all pending logs.
(ucpd-incident-scraper-py3.11) michaelp@MacBook-Air-5 ucpd-incident-scraper %
```
Describe your changes
Added a GoogleMaps fallback for addresses that can't be found in the Census API.
Checklist before requesting a review
make twenty_days output
```terminal (ucpd-incident-scraper-py3.11) michaelp@MacBook-Air-5 ucpd-incident-scraper % make twenty_days python -m incident_scraper days-back 20 Beginning the UCPD Incident scraping process. Finished with the UCPD Incident scraping process. 97 total incidents were scraped from the UCPD Incidents' site. API queries_quota: 60 This incident has a malformed date: {'Incident': 'Information / Burglary', 'Location': 'Kimbark between 56th & 57th', 'Reported': '8/29/223 10:36 AM', 'Occurred': '8/28/23 to 8/29/23 6:00 PM to 8:45 AM', 'Comments / Nature of Fire': 'Two bicycles taken from off-campus residential garage / Referred to CPD', 'Disposition': 'Referred', 'UCPD_ID': '2023-030093'} 1 of 30 contained malformed or voided information. 0 of 30 could not be processed by the Census or GoogleMaps' Geocoder. 29 of 30 incidents were successfully processed. Adding 29 of 30 incidents to the GCP Datastore. Completed adding 29 of 30 incidents to the GCP Datastore. This incident contains voided information: {'Incident': 'Void', 'Location': 'Void', 'Reported': 'Void', 'Occurred': 'Void', 'Comments / Nature of Fire': 'Number generated in error', 'Disposition': 'Void'} This incident contains voided information: {'Incident': 'Void', 'Location': 'Void', 'Reported': 'Void', 'Occurred': 'Void', 'Comments / Nature of Fire': 'Number generated in error', 'Disposition': 'Void'} This incident contains voided information: {'Incident': 'Void', 'Location': 'Void', 'Reported': 'Void', 'Occurred': 'Void', 'Comments / Nature of Fire': 'Number generated in error', 'Disposition': 'Void'} 3 of 30 contained malformed or voided information. 0 of 30 could not be processed by the Census or GoogleMaps' Geocoder. 27 of 30 incidents were successfully processed. Adding 27 of 30 incidents to the GCP Datastore. Completed adding 27 of 30 incidents to the GCP Datastore. 0 of 30 contained malformed or voided information. 0 of 30 could not be processed by the Census or GoogleMaps' Geocoder. 30 of 30 incidents were successfully processed. Adding 30 of 30 incidents to the GCP Datastore. Completed adding 30 of 30 incidents to the GCP Datastore. 0 of 7 contained malformed or voided information. 0 of 7 could not be processed by the Census or GoogleMaps' Geocoder. 7 of 7 incidents were successfully processed. Adding 7 of 7 incidents to the GCP Datastore. Completed adding 7 of 7 incidents to the GCP Datastore. 4 of 97 incidents were NOT added to the GCP Datastore. Program shutting down, attempting to send 5 queued log entries to Cloud Logging... Waiting up to 5 seconds. Sent all pending logs. (ucpd-incident-scraper-py3.11) michaelp@MacBook-Air-5 ucpd-incident-scraper % ```