openstates / openstates-scrapers

source for Open States scrapers
https://openstates.org
GNU General Public License v3.0
846 stars 462 forks source link

[CT,GA,KY,ME,MN,MO,NY,PR,WI] All 2017 individual roll call votes missing for these chambers: #1903

Closed estaub closed 6 years ago

estaub commented 7 years ago

Last update: November 10th. Other chambers were not evaluated. I only re-analyzed the ones that had no resolved 2017 roll calls on Sept 3 to provide more detail. I will update individual issues shortly.

By my reckoning, these chambers currently have, at best, no individual roll call votes with leg_id's recorded in 2017. Some manual work was involved in putting this together; there may be errors. Relevant state-specific Github issues are listed where they exist.

State Issues upper lower
ct #1803 no leg_ids
ga #1360, #1812 no leg_ids no leg_ids
ky #1542 needs scraper needs scraper
me #1865 no leg_ids no leg_ids
mn #1906, #1959 no votes at all; votes are in the upstream calendar; see #1959 for possible cause
mo #1907 no *_votes; needs roll call scraper
ny #1384, #1831, #1960 no votes at all
pr #1908 no *_votes; needs roll call scraper no *_votes; probably unfeasible (without OCR)
wi #1909 no *_votes; votes are visible upstream

Legend

There are several other states with partial failures in this area, like https://github.com/openstates/openstates/issues/1858 and https://github.com/openstates/openstates/issues/1853, that I haven't noted here.

I wrote up a similar issue, with mostly different states, in March: https://github.com/openstates/openstates/issues/1540

FWIW: I haven't kept metrics, but I believe there's been a drastic decrease in quality in roll calls over the last year. I'm not finding fault with anyone, just observing; I know there are reasons!

@jamesturk If after a bit of verification, if you want me to spin off separate issues for { mn, mo, pr, wi }, as I did last time, let me know.

jamesturk commented 7 years ago

Thanks for this, leg_id matching is often a manual process and we haven't done any for these states as we move towards more automated solutions. I'll look into what can be done, as some of these are relatively minor and could probably get some fixes.

jamesturk commented 7 years ago

new issues for the ones that don't already have issues filed would be best, thanks!

estaub commented 7 years ago

Updated original post to reference single-state issues for MN, MO, PR, WI.

jamesturk commented 7 years ago

thanks, closing this in favor of those

jamesturk commented 6 years ago

reopening since there are edits here. we'll triage these in coming weeks

estaub commented 6 years ago

The list of states that are "now" failing in both chambers in this way looks like:

This is based on the most recent session in which there are votes. If there are votes in the session but none have resolved legislators, the state is considered failing. This could be wrong where the 2018 session has opened but there haven't been any roll calls yet.

Also, the Minnesota Senate and New York Assembly are probably still broken (see above).

jamesturk commented 6 years ago

do you have a breakdown of which of these we have votes for (& are unmatched?) and which we're missing votes for? I can run a query this afternoon to do that but if you have it handy that'd be great.

edit: just ran this while doing some other DQ checks, will post issues

estaub commented 6 years ago

Closing; replaced by #2312.