Add Party and Office metadata loaders

[zstumgoren]

Create openelex/us/fixtures directory and add CSV fixtures for

• office names
• parties (start with Dem and Rep)

Let's keep things simple and just create CSVs that can be loaded by standard mongoimport shell command.

• Write shell command to import fixtures using mongoimport. Use the upsert flag and set natural IDs for the upsertFields param (e.g. party abbrev for Party. state/office and district) for office.
• Add these commands to our bootstrap instructions for core repo
• Update Contributing Code docs to note that new parties/offices should be added to appropriate fixtures (pass any doc updates to @dwillis who is working on transforming our GDoc to markdown).

[ghing]

@zstumgoren, do the CSV fixtures in e6b9d52 look right in terms of what you were thinking?

A few thoughts, mostly for documentation. I don't want to get bogged down in analysis paralysis:

• For offices, perhaps it might be better to have per-state fixtures as they'd be easier to maintain/check.
• Load state legislature offices from http://sunlightlabs.github.io/openstates-api/index.html

[zstumgoren]

Haven't checked commit yet, but csvs should mirror the new party and office models on raw result branch.

Let's keep single global offices fixture. It's useful to be able to eyeball them in one place to ensure we apply similar naming conventions across states.

Let's punt on open states api integration for now, but create a placeholder issue to remind us to assess in future.

[ghing]

@zstumgoren, #64 is a pull request into the rawresult branch for this. Once you review and merge I'll update the docs for @dwillis.

[ghing]

@zstumgoren, I'm also a little confused about how this works with EmbeddedDocument models. Won't those be replicated for each of their parent documents? In that case, you can't really access the pre-loaded office and party data from MongoEngine. I guess one way to do it would be to create a EmbeddedOffice class that inherits from both Office and EmbeddedDocument and then just copy over the fields from Office to an embedded EmbeddedOffice doc in your parent doc. Let me know if I'm misunderstanding things.

[zstumgoren]

Woops. I made party and office embedded documents but intended to make them Document subclasses referenced from the Contest model. Otherwise those data loader commands don't make much sense, right? Sorry for the mixup. Feel free to change the parent model in each of those classes on rawresult or i'll make the change later tonight.

[ghing]

@zstumgoren. Just glad I'm understanding things right. You can go ahead and make that change since you've been hacking on the rawresult branch. It shouldn't affect the code I pushed today one way or another. Let me know if I need to make any more changes once you've had a chance to review my pull request.

[zstumgoren]

@ghing Damn, I forgot to add a chamber field on Office that we'll need to populate in the CSV. The name field, which currently appears to contain the chamber, should be populated using a standardized office name. For example, U.S. Senate, U.S. House of Representatives, House of Delegates, State Senate, etc. The chamber field should simply have upper or lower if appropriate (e.g. House of Delegates is lower and State Senate is upper in MD.)

party looks good, though let's use the noun form for name ("Democrat" and "Republican") instead of adjective.

[ghing]

@zstumgoren I can clean up the CSVs. Where can I find the standardized list of office names? I just used the second column of https://github.com/openelections/specs/wiki/Office-Name.

[zstumgoren]

There's no single list that I'm aware of. The names for our target offices will have to be standardized as we add each state. Some can be cribbed from open states, as you pointed out, but for now let's use the list @dwillis created as our starting point. I can add a state field (we'll use 'us' as state for president) and split out the chamber into a separate field. I can make related model changes as well.

[ghing]

@zstumgoren, check out 1cc5aad for fixes to names and adding chamber field.

[zstumgoren]

@ghing nice. looks perfect. thanks!

[zstumgoren]

@ghing just realized you're making these changes on your fork. Can you merge into dev or rawresults branch on openelex when you have a chance? or I can add you as a remote if you're clocked out for the night :)

[ghing]

Just merged PR #64 into rawresult.

[ghing]

@zstumgoren, is this ready to be closed?

[ghing]

@zstumgoren, the docs portion has a pull request in openelections/docs#1. Let me know if this looks right to you.

[dwillis]

I just checked it and it looks good to me, so I merged.