Hawaii is missing roll call votes

[jamesturk]

roll call votes are irregular, impossible to reliably parse and not included in our data

this is noted on http://openstates.sunlightlabs.com/status/

migrated over from Google Code -- http://code.google.com/p/openstates/issues/detail?id=216

[jamesturk]

verified that anyone not listed voted yea, might be possible to support this kind of vote

[jsullivanlive]

Turk says: It's because HI only lists the Nays, not the Yays

[paultag]

@jamesturk is this a thing we should / could turn into a data exception?

[paultag]

Eeek, nevermind, it's state-wide.

[SustainHawaii]

I just found http://HawaiiVotes.org , who copied roll call vote information directly from the Hawaii Legislature website. Apparently the site and process was assisted by Mackinac Center for Public Policy and the "pajama ladies" doing manual, copy-paste transcription.

Missed votes data exists as well... http://hawaiivotes.org/MissedVotes.aspx , which should improve the accuracy of the yes votes, correct?

I assume if Hawaii Votes' manual, copy-paste process directly from http://capitol.hawaii.gov can accomplish these ends, then so can screen-scraping?

[jamesturk]

@SustainHawaii as discussed, we're not entirely comfortable providing data that the state doesn't provide, we have to manage a huge amount of data and relying on third parties/manual sources isn't something that we can do on top of the 100+ scrapers we maintain already

That said, Open States doesn't have to be the end all for your needs, it should be trivial for you all to figure out the no votes given the method you've described (more trivial than for us in fact as we'd need to build in HI-specific exemptions), is there a specific reason why it'd have to be upstream for you?

We'll reconsider if there's a compelling need, but I'm inclined to say that until the HI legislature gets its act together we're not going to have 100% complete votes.

[SustainHawaii]

Just to clarify, the State of Hawaii does provide the information mentioned, and Hawaii Votes is just going to the State of Hawaii site to copy and paste it into their own site.

I do understand that you have plenty more fish to fry, though.

[jamesturk]

Oh perhaps I misunderstood then, the Yes votes are on the HI site now? If so and you can point us towards them I'm sure we can resolve this.

On Mon, Aug 27, 2012 at 2:37 PM, SustainHawaii notifications@github.comwrote:

Just to clarify, the State of Hawaii does provide the information mentioned, and Hawaii Votes is just going to the State of Hawaii site to copy and paste it into their own site.

I do understand that you have plenty more fish to fry, though.

— Reply to this email directly or view it on GitHubhttps://github.com/sunlightlabs/openstates/issues/43#issuecomment-8065765.

[jamesturk]

I just noted on another issue (#398) that Oregon now also does this, if it is something that occurs in at least two states perhaps it is worth the effort to try and come up with a generic solution.

[SustainHawaii]

Here's a link to a Senate Bill's measure status - http://www.capitol.hawaii.gov/measure_indiv.aspx?year=2012&billtype=SB&billnumber=2748

I'm not sure if this is what's being scraped or not, but it looks like each Measure Status page has "Ayes, Ayes with Reservations, Noes and Excused" for each committee member.

"The committee(s) on CPN recommend(s) that the measure be PASSED, UNAMENDED. The votes in CPN were as follows: 4 Aye(s): Senator(s) Baker, Taniguchi, Solomon; Aye(s) with reservations: Senator(s) Slom ; 0 No(es): none; and 3 Excused: Senator(s) Galuteria, Green, Nishihara."

[jamesturk]

Those only exist for committee votes (which we collect) - you'll notice the COW votes don't include the Ayes.

Passed Final Reading, as amended (CD 1). 25 Aye(s); Aye(s) with reservations: none . 0 No(es): none. 0 Excused: none.

On Tue, Aug 28, 2012 at 2:00 PM, SustainHawaii notifications@github.comwrote:

Here's a link to a Senate Bill's measure status - http://www.capitol.hawaii.gov/measure_indiv.aspx?year=2012&billtype=SB&billnumber=2748

I'm not sure if this is what's being scraped or not, but it looks like each Measure Status page has "Ayes, Ayes with Reservations, Noes and Excused" for each committee member.

"The committee(s) on CPN recommend(s) that the measure be PASSED, UNAMENDED. The votes in CPN were as follows: 4 Aye(s): Senator(s) Baker, Taniguchi, Solomon; Aye(s) with reservations: Senator(s) Slom ; 0 No(es): none; and 3 Excused: Senator(s) Galuteria, Green, Nishihara."

— Reply to this email directly or view it on GitHubhttps://github.com/sunlightlabs/openstates/issues/43#issuecomment-8100938.

[SustainHawaii]

In this example below I can see there are no Aye(s) explicitly written:

Passed Final Reading as amended in CD 1 with none voting aye with reservations; none voting no (0) and Representative(s) Ching excused (1).

Sorry for being coding-obtuse, but if I'm understanding correctly, the challenge is because the Aye votes are not explicit to screen scrape, so more code is required to "pull the Aye(s)" by identifying the total members voting in this hearing and removing the ones that are listed from the total list of Aye(s) with Reservations, Noes and Excused?

[jamesturk]

Right, if a member is absent for example it is unclear if they voted Yes or not at all.

On Tue, Aug 28, 2012 at 2:19 PM, SustainHawaii notifications@github.comwrote:

In this example below I can see there are no Aye(s) explicitly written:

Passed Final Reading as amended in CD 1 with none voting aye with reservations; none voting no (0) and Representative(s) Ching excused (1).

Sorry for being coding-obtuse, but if I'm understanding correctly, the challenge is because the Aye votes are not explicit to screen scrape, so more code is required to "pull the Aye(s)" by identifying the total members voting in this hearing and removing the ones that are listed from the total list of Aye(s) with Reservations, Noes and Excused?

— Reply to this email directly or view it on GitHubhttps://github.com/sunlightlabs/openstates/issues/43#issuecomment-8101576.

[SustainHawaii]

For our purposes, can we just have a qualifier on the site that says "absent vote" or "excused" means "no vote"?

[SustainHawaii]

Just curious what the status was on this issue... does it look like something you folks will be able to address or is it too customized and best left to a Hawaii interest to address?

[jamesturk]

It is probably best left to Hawaii but there is still some small amount of hope, investigating a solution that would make it possible to fix this for HI and OR (the two states that don't include Yes votes)

On Fri, Sep 7, 2012 at 2:25 PM, SustainHawaii notifications@github.comwrote:

Just curious what the status was on this issue... does it look like something you folks will be able to address or is it too customized and best left to a Hawaii interest to address?

— Reply to this email directly or view it on GitHubhttps://github.com/sunlightlabs/openstates/issues/43#issuecomment-8374143.

[bshor]

Is this fixed in OR and HI?

[jamesturk]

Sorry, should have included a message. It hasn't but the issue list was woefully out of date as it had been split across two ticketing systems for a while and a lot of stale 2012 issues were around from before the migration to the Sunlight JIRA bugtracker, many of which were closed there. We're going to start fresh and will most likely be opening state-specific issue trackers in coming weeks as we get things up and running again.

On Mon, Nov 7, 2016 at 7:05 PM, Boris Shor notifications@github.com wrote:

Is this fixed in OR and HI?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/openstates/openstates/issues/43#issuecomment-259004359, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAfYpP9S6NLkYGXceNs-M0_ojoaLECUks5q77zigaJpZM4AEfwI .