search

openstates / openstates-scrapers

source for Open States scrapers
https://openstates.org
GNU General Public License v3.0
846 stars 464 forks source link

VI: Write Legislator Scraper #1344

Closed showerst closed 6 years ago

showerst commented 7 years ago

State: VI

VI needs a legislator scraper. This would be a nice starter project if someone wants to jump in to openstates.

You could scrape the names from the main menu, the photos and legislator URLs from their legislator pages, and the capitol/district office contact info from the contact us pages.

squeakyspacebar commented 7 years ago

I've made some headway into this, but I'm unable to find a decent source for the legislator parties. Anybody have any ideas?

mshenfield commented 7 years ago

I wasn't able to find the senator's parties on http://www.legvi.org, except for mentions in some profiles, e.g. http://www.legvi.org/index.php/senator-jean-a-forde

he is a member of the St. Thomas District Democratic Party

I was able to find party info online, though I'm not sure where that information was gathered from.

hiteshgarg14 commented 7 years ago

I would like to write Legislator Scraper. As information about legislators is not sufficient on http://www.legvi.org. Should I write scrapper using wikipedia and legvi pages or just head over to another issue?

jamesturk commented 7 years ago

hmm, I think the best solution here would be to hardcode a dictionary of names->party

it isn't optimal but if it'll let this proceed I'm all for it

since multiple people have looked at this it might be good if any of you w/ a branch in progress would post a WIP PR so others can review/work on it

jamesturk commented 6 years ago

figured this belongs over here since it'll be mostly manual: https://github.com/openstates/people/issues/27