search

City-Bureau / city-scrapers-atl

City Scrapers for Atlanta
MIT License
1 stars 0 forks source link

🕷️ Fix spider mixin: Cobb County #157

Closed SimmonsRitchie closed 7 months ago

SimmonsRitchie commented 7 months ago

What's this PR do?

Fixes our Cobb County mixin which is used by atl_cobb_county_elections and atl_cobb_county_boc

Why are we doing this?

Changes to page structure appear to have caused the current implementation to break.

Steps to manually test

After installing the project using pipenv:

  1. Activate the virtual environment:

    pipenv shell

  2. Run the spider:

    scrapy crawl atl_cobb_county_elections -O test_output.csv

  3. Monitor the stdout and ensure that the crawl proceeds without raising any errors. Pay attention to the final status report from scrapy.

  4. Inspect test_output.csv to ensure the data looks valid. I suggest opening a few of the URLs under the source column of test_output.csv and comparing the data for the row with what you see on the page.

Are there any smells or added technical debt to note?