City-Bureau / city-scrapers-atl

City Scrapers for Atlanta
MIT License
1 stars 0 forks source link

New Scraper: Cobb County Elections and Registration #16

Closed ghost closed 2 years ago

ghost commented 2 years ago

Create a new scraper for Cobb County Elections and Registration

Website: https://www.cobbcounty.org/events?field_section_target_id=All&field_event_category_target_id=195&field_event_date_recur_value_2=&field_event_date_recur_end_value= Jurisdiction: Cobb County Classification: Elections Board is empowered with all powers and duties relating to the conduct of elections as election superintendents and the registration of voters and absentee balloting procedures as board of registrars as provided by Ga. Law.

rhetr commented 2 years ago

i found that this site exposes the Drupal JSON:API, which can be filtered as such.

here is a query that should return all the events tagged as a "public meeting" happening on or after the current date:

https://www.cobbcounty.org/jsonapi/node/event
?filter[field_event_category.meta.drupal_internal__target_id]=195
&filter[or-group][group][conjunction]=OR
&filter[date][condition][path]=field_event_date.value
&filter[date][condition][operator]=%3E%3D
&filter[date][condition][value]=2022-08-11
&filter[date][condition][memberOf]=or-group
&filter[daterecur][condition][path]=field_event_date_recur.value
&filter[daterecur][condition][operator]=%3E%3D
&filter[daterecur][condition][value]=2022-08-11
&filter[daterecur][condition][memberOf]=or-group

similar queries may or may not work for other drupal sites--drupal is not necessarily as predictable as wordpress, so it remains to be seen whether the code written for this scraper is generalizable to others.

ghost commented 2 years ago

Nice find! Out of curiosity (so I can add it to my own checks) how'd you come across it?

Glad to update the Cobb County scrapers to use this.

& yeah we can wait on generalizing it until/unless we find more Drupal JSON API

rhetr commented 2 years ago

i was able to find that drupal sites' default path to access the JSON:API is at /jsonapi, so i gave it a shot and it worked on this website 🙂 it doesn't look like jsonapi is enabled for every drupal site though, eg the dekalb county website uses drupal as well but doesn't show anything at /jsonapi--not sure if it's just totally disabled or if they somehow changed the path.

ghost commented 2 years ago

actual URL is https://www.cobbcounty.org/elections/events -- I haven't done the JSON API part yet as I wasn't seeing the times & needed to dig deeper to make sure they were in there. I did however add a scraper for this and factored out the common Cobb Co. stuff so if/when we replace w/ JSON API it can be in one place