search

ScrapeOpen / elezionistorico.interno.gov.it

Scraper for Historical archive of Italian elections published by the Italy's Ministry of Interior.
MIT License
0 stars 0 forks source link

Anyone interested in writing code to scrape elections held before 2018? #1

Open fraba opened 6 years ago

fraba commented 6 years ago

The code in scrape_20180304.py can be easily used to scrape other elections, but it must its universality must be improved.

matg19 commented 5 years ago

Hi!

This scraper is awesome! I am currently working on a paper and I needed results at the comune level starting at earliest in the 20th century as possible. This worked like magic so I am happy to help with what I did.

The code only needed some minor change to be able to scrape all the elections. This is my approach, I noticed it makes it a bit slower than the original one, there might be a better way of solving it but so far is working great.

In the preamble, require:

from selenium.common.exceptions import NoSuchElementException And in the scrapeItalia function, instead of setting the level 6 as default let it choose the level conditional on receiving the NoSuchElementException from selenium (first the lowest, then a higher one):

driver.get(level_5_href) try: downloadCSV(6) except NoSuchElementException: downloadCSV(5)

fraba commented 5 years ago

Glad it helps!

It would be great if you could contribute to the repository the actual script you used for each election and also share the data collected.

F

On Sat, 23 Feb 2019 at 8:00 am, matg19 notifications@github.com wrote:

Hi!

This scraper is awesome! I am currently working on a paper and I needed results at the comune level starting at earliest in the 20th century as possible. This worked like magic so I am happy to help with what I did.

The code only needed some minor change to be able to scrape all the elections. This is my approach, I noticed it makes it a bit slower than the original one, there might be a better way of solving it but so far is working great.

In the preamble, require:

from selenium.common.exceptions import NoSuchElementException And in the scrapeItalia function, instead of setting the level 6 as default let it choose the level conditional on receiving the NoSuchElementException from selenium (first the lowest, then a higher one):

driver.get(level_5_href) try: downloadCSV(6) except NoSuchElementException: downloadCSV(5)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ScrapeOpen/elezionistorico.interno.gov.it/issues/1#issuecomment-466545995, or mute the thread https://github.com/notifications/unsubscribe-auth/AD-WGcWxV3y_lR27SceWIIXhrcSUbNWyks5vQFpbgaJpZM4UX277 .

-- m: +61 416 860 221 e: bailo.francesco@gmail.com