search

openelections / openelections-core

Core repo for election results data acquisition, transformation and output.
MIT License
179 stars 98 forks source link

Washington transforms and validators #219

Open ghing opened 10 years ago

ghing commented 10 years ago

Write the transforms and validators for Washington.

Fundamentally there should be a transform to create Candidate, Contest and Result records from the RawResult records created in the loader. Use best judgement if you want to delegate this to multiple transform classes or functions. If you encounter ambiguities or the need to standardize things in the data, these will require transforms as well.

Write validators to check values in results and whether aggregate queries match published racewide totals. I took a TDD approach and tried to write these before the transforms, but do whatever works for you.

Maryland is the only state that has transforms and validators so they're a reasonable reference.

Since these have only been implemented for one state so far, I'm open to feedback on the API.

ericlagergren commented 10 years ago

Well the first issue I ran into is the mongoengine query set can't find any data -- even Maryland's transform returns openelex.models.DoesNotExist: Office matching query does not exist.

ghing commented 10 years ago

You probably need to run inv load_metadata.run --collection=office first.

ericlagergren commented 10 years ago

@ghing Oh, gotcha. So I need to add the canonical names first before I write the transform portion? Because I started on the transform two days ago, but I ran into this roadblock last night and not even #python or #mongodb or mongoengine's docs could help me.

ghing commented 10 years ago

@EricLagerg Yeah, you'll need to prepopulate Office and Party records to be able to fill in the relation fields on the other models.

ericlagergren commented 10 years ago

Gotcha also add a call at the end of why 96 on the Washington's loader module you're missing it and it breaks the loader :)

Eric Lagergren ericlagergren.com

On Tue, Oct 14, 2014 at 9:08 PM, Geoffrey Hing notifications@github.com wrote:

@EricLagerg Yeah, you'll need to prepopulate Office and Party records to be able to fill in the relation fields on the other models.

Reply to this email directly or view it on GitHub: https://github.com/openelections/core/issues/219#issuecomment-59155296

ghing commented 10 years ago

@EricLagerg done in 75a73dc9bcc3f583f295a9a37b21071ebbab7c51. Good catch.

ericlagergren commented 9 years ago

I'm really sorry I've been MIA since the end of last year. Long story short, life's been a bit hectic. I still intend to get this done for you guys as soon as I have a free couple days to sit down and dive back into it. Anyway, again, sorry. @ghing @dwillis

dwillis commented 9 years ago

We've all been busy, so don't feel too bad. Let us know when you're back in the swing of things!