Critical issue: stats aren't adding up

mattmaiorana commented 11 years ago

Jeff Miller (CA-31) says “Received $32,850” but when you add up 2008, 2010, and 2012 figures it would equal $58,750. It’s also not equal to the most recent year ??? -- Should show the lifetime total, I think.

Also the case for Michael Rubio, etc. Think this is widespread bug.

Same with companies yearly vs. “Contributed”

CSV file seems to reflect the lower number - should also be updated

michalgm commented 11 years ago

I'm fixing this now - this happened for a few different reasons: 1) When importing the data, I forgot to delete contributions from companies that are sometimes coded as oil & gas, but we know never should be, such as Seven Eleven. These were properly ignored by the graph generation, but not in the chart reporting. Now they are being removed from the database, so it shouldn't be an issue. 2) I also forgot to remove contributions to PACs (such as Republican Senate Committee). Normally, this wouldn't be an issue, except that for some reason the source data assigns the same id to companies and people

this is why Donald Olson (AK-Senate-12) numbers were so wildly off - he shared an id with the Pennsylvania Republican Party. Anyway, I'm now deleting all contributions to PACs as well, so this won't be a problem anymore. 3) In addition to the above two issues, the company report was off because the original design spec said that the charts should be static no matter where you were looking at them - i.e. they pulled data across all states/chambers/years. We've decided to change that so that now they will only pull from the state/chamber you are viewing.

I will close this issue once all the appropriate code is in place on the live server.

On 07/31/2013 06:22 PM, mattmaiorana wrote:

Jeff Miller (CA-31) says “Received $32,850” but when you add up 2008, 2010, and 2012 figures it would equal $58,750. It’s also not equal to the most recent year ??? -- Should show the lifetime total, I think.

Also the case for Michael Rubio, etc. Think this is widespread bug.

Same with companies yearly vs. “Contributed”

CSV file seems to reflect the lower number - should also be updated

— Reply to this email directly or view it on GitHub https://github.com/michalgm/state_dem/issues/63.

michalgm commented 11 years ago

This is mostly fixed - there may still be some random edge cases, but pretty rare. I've started building out an infrastructure to identify these inconsistencies, but it will have to wait until phase 2.

michalgm / state_dem

Critical issue: stats aren't adding up #63