This code is going to be used to scrape the UCPD Daily Incident page at a pre-determined frequency and store the incidents on a generic JSON data-store.
MIT License
3
stars
2
forks
source link
Add Census geocoder and combine geocoder classes #44
Added the Census geocoder back because Google was costin' me a lot of money.
Checklist before requesting a review
[x] The code runs successfully.
(ucpd-incident-scraper-py3.11) michaelp@MacBook-Air-18 ucpd-incident-scraper % make thirty_days
python -m incident_scraper days-back 30
Beginning the UCPD Incident scraping process.
Finished with the UCPD Incident scraping process.
156 total incidents were scraped from the UCPD Incidents' site.
API queries_quota: 60
This incident has an insufficient number of keys: {}
Incident type changed from Found Narcotics to Found Narcotic.
Incident type changed from Found Narcotics to Found Narcotic.
Incident type changed from Information / Attempt Theft to Information / Attempted Theft.
Incident type changed from Found Narcotics to Found Narcotic.
Incident type changed from Information / Attempt Armed Robbery to Information / Attempted Armed Robbery.
This incident has an insufficient number of keys: {}
Incident type changed from Deceptive Practice / Fraudulent Checks to Deceptive Practice / Fraudulent Check.
2 of 100 contained malformed or voided information.
0 of 100 could not be processed by the GoogleMaps' Geocoder.
98 of 100 incidents were successfully processed.
Adding 98 of 100 incidents to the GCP Datastore.
Completed adding 98 of 100 incidents to the GCP Datastore.
Incident type changed from Information / Attempt Robbery to Information / Attempted Robbery.
Incident type changed from Found Narcotics to Found Narcotic.
Incident type changed from Found Narcotics to Found Narcotic.
Incident type changed from Information / Attempt Burglary to Information / Attempted Burglary.
Incident type changed from Attempt Armed Robbery to Attempted Armed Robbery.
0 of 56 contained malformed or voided information.
0 of 56 could not be processed by the GoogleMaps' Geocoder.
56 of 56 incidents were successfully processed.
Adding 56 of 56 incidents to the GCP Datastore.
Completed adding 56 of 56 incidents to the GCP Datastore.
2 of 10 'Information' incidents predicted into other categories.
2 of 156 incidents could NOT be added to the GCP Datastore.
Program shutting down, attempting to send 3 queued log entries to Cloud Logging...
Waiting up to 5 seconds.
Sent all pending logs.
(ucpd-incident-scraper-py3.11) michaelp@MacBook-Air-18 ucpd-incident-scraper %
Describe your changes
Added the Census geocoder back because Google was costin' me a lot of money.
Checklist before requesting a review