search

sunlightlabs / read_FEC

Turn raw electronic FEC filings into meaningful data
http://realtime.influenceexplorer.com
BSD 3-Clause "New" or "Revised" License
19 stars 7 forks source link

Independent expenditures being double-counted in some cases #139

Open rshorey opened 8 years ago

rshorey commented 8 years ago

For example see http://realtime.influenceexplorer.com/committee/2016/florida-first-project/C00620476/

(compared to https://projects.propublica.org/itemizer/committee/C00620476/2016 - the propublica number is correct)

I believe the problem is that independent expenditures that come from 24/48 hour report forms (F24s) need to be deleted when a corresponding quarterly filing comes in. Newer versions of F24s don't always fill out both expenditure_date and dissemination_date fields, so you have to check both to know whether to deactivate the given IE.

jsfenfen commented 8 years ago

Yeah, I think this is an indirect result of an introduction of an additional date field in the 8.0-8.1 (?) shift; originally there was only a single date allowed, see here. This may require the second date to get it's own _formatted field (I think the default is to store dates as strings and only convert the ones that are needed).

jsfenfen commented 8 years ago

Nope, the second formatted field is already there--this just needs a query tweak, I think. Fixing the past, though, would likely require running this check on all F3X's with >0 ies present. Hmm.