We know that our scraper works for San Jose, but we should begin testing it out on other cities that also use the Legistar platform. Please determine what, if any, different parameters / arguments need to be included in order to successfully scrape other cities that use Legistar.
Tasks:
[ ] Design a separate set of launch.json arguments to use the Legistar_Selenium scraper on Sunnyvale, Mountain View and any other cities that you want to try. See here for a list of all the Legistar cities in California.
[ ] Create a spreadsheet of any errors that occur during the scrape and / or issues.
Note: in order to scrape from cities besides San Jose, you will likely need to change the link text search in the Legistar_selenium scraper from "Memorandum" to "Staff Report". San Jose labels its staff memos as "Memorandum" whereas most other cities call them "Staff reports"
We know that our scraper works for San Jose, but we should begin testing it out on other cities that also use the Legistar platform. Please determine what, if any, different parameters / arguments need to be included in order to successfully scrape other cities that use Legistar.
Tasks:
[ ] Design a separate set of launch.json arguments to use the Legistar_Selenium scraper on Sunnyvale, Mountain View and any other cities that you want to try. See here for a list of all the Legistar cities in California.
[ ] Create a spreadsheet of any errors that occur during the scrape and / or issues.
Note: in order to scrape from cities besides San Jose, you will likely need to change the link text search in the Legistar_selenium scraper from "Memorandum" to "Staff Report". San Jose labels its staff memos as "Memorandum" whereas most other cities call them "Staff reports"
Line 142 in Legistar_Selenium:
memorandum = driver.find_element_by_link_text('Memorandum')