influence-usa / lobbying_federal_domestic

Public domain data collectors for federal domestic lobbying disclosures. https://github.com/unitedstates-influence/lobbying-federal-domestic/wiki
Other
11 stars 5 forks source link

Transform step for incomes is making a mistake #14

Closed zmaril closed 10 years ago

zmaril commented 10 years ago

If you compare the original copy of this document to the transformed data, it seems that there is a mistake in how the various fields related to expense are calculated. Specifically, the expense_less_than_five_thousand field is false, implying that the expense is greater than 5000, while the expense_amount field is null. In the original document, neither field was check and expenses was left blank, which seems to imply that there were no expenses (rather than implying there was an unknown expense that was greater than 5000).

boblannon commented 10 years ago

yeah, looks like there's some problem with the boolean checkbox parser.

boblannon commented 10 years ago

Okay, I see the issue. The filer just left the expenses field completely blank, which isn't a case I'd planned for. For both income and expense, if both of these boxes are unchecked, the value of expense_less_than_five_thousand should probably just be null.

screenshot from 2014-07-24 10 33 14

zmaril commented 10 years ago

Makes sense to me.

boblannon commented 10 years ago

fixed in deee6ef