michplunkett / ucpd-incident-scraper

This code is going to be used to scrape the UCPD Daily Incident page at a pre-determined frequency and store the incidents on a generic JSON data-store.
MIT License
3 stars 2 forks source link

Add Census geocoder and combine geocoder classes #44

Closed michplunkett closed 9 months ago

michplunkett commented 9 months ago

Describe your changes

Added the Census geocoder back because Google was costin' me a lot of money.

Checklist before requesting a review

(ucpd-incident-scraper-py3.11) michaelp@MacBook-Air-18 ucpd-incident-scraper % make thirty_days
python -m incident_scraper days-back 30
Beginning the UCPD Incident scraping process.
Finished with the UCPD Incident scraping process.
156 total incidents were scraped from the UCPD Incidents' site.
API queries_quota: 60
This incident has an insufficient number of keys: {}
Incident type changed from Found Narcotics to Found Narcotic.
Incident type changed from Found Narcotics to Found Narcotic.
Incident type changed from Information / Attempt Theft to Information / Attempted Theft.
Incident type changed from Found Narcotics to Found Narcotic.
Incident type changed from Information / Attempt Armed Robbery to Information / Attempted Armed Robbery.
This incident has an insufficient number of keys: {}
Incident type changed from Deceptive Practice / Fraudulent Checks to Deceptive Practice / Fraudulent Check.
2 of 100 contained malformed or voided information.
0 of 100 could not be processed by the GoogleMaps' Geocoder.
98 of 100 incidents were successfully processed.
Adding 98 of 100 incidents to the GCP Datastore.
Completed adding 98 of 100 incidents to the GCP Datastore.
Incident type changed from Information / Attempt Robbery to Information / Attempted Robbery.
Incident type changed from Found Narcotics to Found Narcotic.
Incident type changed from Found Narcotics to Found Narcotic.
Incident type changed from Information / Attempt Burglary to Information / Attempted Burglary.
Incident type changed from Attempt Armed Robbery to Attempted Armed Robbery.
0 of 56 contained malformed or voided information.
0 of 56 could not be processed by the GoogleMaps' Geocoder.
56 of 56 incidents were successfully processed.
Adding 56 of 56 incidents to the GCP Datastore.
Completed adding 56 of 56 incidents to the GCP Datastore.
2 of 10 'Information' incidents predicted into other categories.
2 of 156 incidents could NOT be added to the GCP Datastore.
Program shutting down, attempting to send 3 queued log entries to Cloud Logging...
Waiting up to 5 seconds.
Sent all pending logs.
(ucpd-incident-scraper-py3.11) michaelp@MacBook-Air-18 ucpd-incident-scraper %