Add data for alternate roles (Speaker, Pro Tem, Whip)

[jasonab]

I see other issues address this same area (#26 #27), but it would be nice to have data on who is the Speaker, the President Pro Tempore, and other offices.

[dwillis]

Agreed - I've begun adding leadership positions using the new House History IDs to NYT db, and will cover the house leadership positions here. Only question is how we want to store such info - as a role within the current/historic legislators setup, or separately?

[konklone]

These roles are few, and don't change often - I would be fine with curating them manually. The simplest way to add this is to add a leadership or leadership_title field on each term element for which that person has a leadership role.

[dwillis]

There are instances when the same person holds more than one leadership position within the same term (not concurrently, but ascends from one spot to another).

[JoshData]

Do all of these positions reset in each Congress? (i.e., we'd typically have three per Senate role?)

[konklone]

Sigh. Well, I'm also fine simply putting the leadership title at the top-level and holding only their current title. Someone's past leadership roles and timelines are far less important at interpreting historical Congressional activity than their elected office.

[dwillis]

Leadership positions expand and contract, and sometimes change titles. It's super fun. I think if we're going to have leadership roles, we should have them all. Perhaps a leadership node within a term that has one or more positions and an optional end_date?

[konklone]

OK, but instead of adding complexity to the terms array, let's make a leadership_roles array that has its own fields, and start and end dates. Does that work?

[JoshData]

I'm OK with any of those options, but if we go with @konklone's leadership_roles as a top-level thing, this would be a good time to add a 'congress' or 'congresses' field of some sort.

[dwillis]

Ok, so add congress to both leadership_roles array and terms array?

[JoshData]

I just meant leadership_roles for now. Since you probably already have the congress number.

[konklone]

I'm doing a manual add of Pelosi, Boehner, Reid, and McConnell's current leadership roles now, and I'll submit a pull request so we can review the format. I'll add a congresses field.

[konklone]

Filed pull request #68, check it out.

[JoshData]

Am wondering if we want to be more detailed about dates. E.g. The Speaker is re-elected each Congress. So there's no Speaker from swearing in until the completion of the vote, which would suggest splitting the role into terms.

[konklone]

We could split it by any date - the end of a Congress term, or the changing of title mid-term. That would obviate the need for a "congresses" field.

On Wed, Apr 17, 2013 at 9:54 PM, Joshua Tauberer notifications@github.comwrote:

Am wondering if we want to be more details about dates. E.g. The Speaker is re-elected each Congress. So there's no Speaker from swearing in until the completion of the vote, which would suggest splitting the role into terms.

— Reply to this email directly or view it on GitHubhttps://github.com/unitedstates/congress-legislators/issues/67#issuecomment-16548380 .

Developer | sunlightfoundation.com

[konklone]

OK, check out #68 now - I've broken out their leadership terms to not cross congressional session boundaries, and removed the congresses field.

Note that Senate leaders have their leadership terms broken out every 2 years, which is different than their normal terms.

Also, this would handle better the never-yet-happened situation where someone external to Congress is elected Speaker of the House - they could have an empty terms array, but a valid leadership roles field.

[tbruce]

Sorry, guys -- I've got a lot of catching-up to do here, and I don't know that I fully understand the data model. But from what I can see I think it might be worthwhile to model memberships (of any kind -- officeholders, committee roles, whatever) as objects in their own right, with durations and optional roles attached. The use case I've run across that seems to present the critical case is one in which a committee member holds continuous membership on the committee but bops around between roles -- sometimes a plain-vanilla member, sometimes the chairman, sometimes the ranking minority member, etc. Talked a lot about this problem here: http://liicr.nl/103Fk6A . Would such an approach fit with what you're doing? (I confess to not being able to follow the existing model completely)

[konklone]

I think that's the right approach for modeling all role-based activity and history; so far, we haven't decided to really Do Roles Right, though. This issue is focused specifically on roles of House and Senate leadership positions (which fortunately are a lot less fluid), and I think the general mentality here is that the data's high value enough to record, but we should get in and get out with as little disturbance to the existing data as possible.

[dwillis]

Was thinking about this with our NYT data, too, and decided not to create a specific "leadership roles" table but to add leadership roles into our general roles table and then create non-voting member roles to represent leadership positions. We don't really do this for committees - we have one record for each person's service on a committee, so we don't really acknowledge that, for example, Menendez was a member of Senate Foreign Affairs and then its chairman. Anyway, I think we're ok for leadership roles with the approach we've discussed here.

[tbruce]

Mostly for my own amusement, I guess, I'm going to start mapping this stuff into the data model we did for the Anonymous Client. I have a lot of catching up to do, though -- we don't yet have description information entered for most of the model, so fingers will be flying..... and then I have to do the map.

[konklone]

There's a related issue that would necessitate us creating a central output script that would probably run automatically on every push to Github. That output script would generate alternate formats of the data, such as JSON or CSV (the canonical format right now is YAML, for diff-ability and read-ability). That script, once it exists, would be a pretty ideal place to add additional output formats that could at least include greater denormalization.

[konklone]

I've merged in #68. Thanks for working this out with me, guys. @jasonab, does this help with what you need?

[jasonab]

Thanks, Eric, this will be handy. Is sunlight going to be offering this data directly?

[konklone]

I could let this bubble through into the Sunlight Congress API as undocumented fields easily enough, though I'd prefer to see it in production somewhere and get feedback on whether it works, before I formally document it.

On Mon, May 13, 2013 at 2:53 PM, Jason Bennett notifications@github.comwrote:

Thanks, Eric, this will be handy. Is sunlight going to be offering this data directly?

— Reply to this email directly or view it on GitHubhttps://github.com/unitedstates/congress-legislators/issues/67#issuecomment-17832552 .

Developer | sunlightfoundation.com