openstates / openstates-scrapers

source for Open States scrapers
https://openstates.org
GNU General Public License v3.0
846 stars 464 forks source link

Vermont: showing 3 extra House members #2024

Closed benjamin-brown closed 6 years ago

benjamin-brown commented 6 years ago

Looks like you're showing 153 House members when there should only be 150.

I could only find one duplicate (Ruqaiyah Morris in Bennington-2-2)

Not sure about the other two. I'll amend this ticket if I find them.

image

estaub commented 6 years ago

I'd check districts with recent special elections - see if the old rep is still there.

benjamin-brown commented 6 years ago

In addition to the duplicate of Rep. Morris, I found 4 legislators on OS who are not on the VT site, and 2 legislators on the VT site who are not on OS. Starting with 3 too many it looks like the math works:
153 + 2(new) = 155 - 1(dup) = 154 - 4(extra) = 150 = correct # of VT Reps


On VT Leg, not OS:

Christopher Mattos / Chittenden-10 https://legislature.vermont.gov/people/single/2018/29153

James Harrison / Rutland-Windsor-1 https://legislature.vermont.gov/people/single/2018/28878


On OS, not VT Leg:

Ronald Hubert / Chittenden-10 https://openstates.org/vt/legislators/VTL000087/ronald-e-hubert/

Job Tate / Rutland-Windsor-1 https://openstates.org/vt/legislators/VTL000367/job-tate/

Adam Greshin / Washington-7 https://openstates.org/vt/legislators/VTL000077/adam-m-greshin/

Oliver Olsen / Windham-Bennington-Windsor https://openstates.org/vt/legislators/VTL000138/oliver-k-olsen/


Here's the link to Rep. Morris: https://legislature.vermont.gov/people/single/2018/24016

estaub commented 6 years ago

A clean scrape does all of these correctly, afaict, but maybe there's some key field missing. I've compared the Mattos record with a working one and don't see an issue. For Mattos:

people record ```json { "gender": "", "summary": "", "name": "Christopher Mattos", "other_names": [], "_id": "6ca675d4-e745-11e7-90a2-0242ac120003", "national_identity": "", "links": [{"url": "http://legislature.vermont.gov/people/single/2018/29153", "note": ""}], "birth_date": "", "death_date": "", "contact_details": [ { "note": "Capitol Office", "value": "Vermont State House\n115 State Street\nMontpelier, VT 05633", "type": "address" }, {"note": "Capitol Office", "value": "cmattos@leg.state.vt.us", "type": "email"}, {"note": "District Office", "value": "37 Smith Rd.\nMilton, VT 05468", "type": "address"}, {"note": "District Office", "value": "(802) 922-2059", "type": "voice"} ], "image": "https://legislature.vermont.gov/assets/Documents/Legislators/CMattos.jpg", "biography": "", "sources": [ {"url": "http://legislature.vermont.gov/people/loadAll/2018", "note": ""}, {"url": "http://legislature.vermont.gov/people/single/2018/29153", "note": ""} ], "identifiers": [], "extras": {} } ```
person membership record ```json { "person_name": "Christopher Mattos", "start_date": "", "contact_details": [], "_id": "6ca67d2c-e745-11e7-90a2-0242ac120003", "post_id": null, "role": "member", "label": "", "extras": {}, "links": [], "on_behalf_of_id": null, "person_id": "6ca675d4-e745-11e7-90a2-0242ac120003", "end_date": "", "organization_id": "~{\"classification\": \"party\", \"name\": \"Republican\"}" } ```
house membership record ```json { "person_name": "Christopher Mattos", "start_date": "", "contact_details": [], "_id": "6ca67a84-e745-11e7-90a2-0242ac120003", "post_id": "~{\"label\": \"Chittenden-10\", \"organization__classification\": \"lower\"}", "role": "", "label": "", "extras": {}, "links": [], "on_behalf_of_id": null, "person_id": "6ca675d4-e745-11e7-90a2-0242ac120003", "end_date": "", "organization_id": "~{\"classification\": \"lower\"}" } ```

I had a great theory that they were only messing up in districts with two representatives, but the last one, Oliver Olsen, is a solo rep.

benjamin-brown commented 6 years ago

Thanks @estaub - however I'm not an engineer so the code there is over my head.

RE: Rep. Oliver Olsen - His replacement was just named today, so I'm sure that new entry will appear in VT's legislative website soon.

I also noticed OpenStates is now showing only two extra legislators, but I'm unsure if anyone has made any changes yet (this ticket isn't assigned to anyone) or if something has just resolved on it's own. https://openstates.org/vt/

Pardon the ignorance here, but I'm unsure whether you're comment is suggesting I make the edits myself (which regrettably I'm unable to do), or you're suggesting some changes to whomever may be assigned to this ticket at some point, or you're offering to make some changes yourself?

Sorry for the confusion, trying to figure out next steps...

estaub commented 6 years ago

@benjamin-brown I'm a junior contributor, went as far as I can with it, and wanted to leave notes for whoever ends up picking it up.

benjamin-brown commented 6 years ago

Thanks for the clarification, @estaub ! And for the help!

benjamin-brown commented 6 years ago

Just noticed OpenStates is now showing two extra Senators, as well as two extra House members.

image

Looks like Randy Brock (just recently appointed) has a duplicate, and the Sen he replaced, Dustin Degree, still has an active profile.

image

image

jamesturk commented 6 years ago

@mileswwatkins got this one fixed!