search

fecgov / fec-cms

The content management system (CMS) for the new Federal Election Commission website.
https://www.fec.gov
Other
93 stars 38 forks source link

Check raising and spending breakdown charts #1983

Closed patphongs closed 6 years ago

patphongs commented 6 years ago

We need to verify the accuracy per cycle the data that is displayed in these charts:

Raising breakdown: https://fec-stage-proxy.app.cloud.gov/data/raising/

Spending breakdown: https://fec-stage-proxy.app.cloud.gov/data/spending/

Related issue #1949

jwchumley commented 6 years ago

I compared the candidates and monetary amounts between stage and fec.gov and with the data in FECP and they all agree for the three two year periods in production - 1012, 2016 and 2018. I also compared 2008 on stage with the data in fecp and it matches up perfectly.

llienfec commented 6 years ago

Received feedback from a user about this graph - saying the numbers looked off and the interaction isn't working correctly. It looks like Jeff verified the numbers above. Here are 5 issues that I found as I tried to research the numbers and how the graph worked. I think these 5 issues are eroding trust in the numbers:

image

  1. We've noted that the scale of the map ($4 billion) doesn't work for the off-election year because it looks like no committees are raising money in 2017. More than that, the singe orange line makes it look like only PACs are represented in the map, even though the key shows that candidates and parties should also show up.
  2. When the user moves the mouse over the months of the year on the x-axis, the line moves on the month to the right of the mouse. The user expected the mouse and the line to move together.
  3. The user was looking for a line showing the "all committees" totals on the graph since it was listed in the key first, especially since this number is big enough to appear on the graph's scale in the off-election year.
  4. When trying to verify numbers, I tried to go to the "methodology" button. When the screen pops up, you can't scroll down. If your screen is small (verified on my tablet), you can't see all of the methodology information.
  5. PAC data - January and May 2017 cumulative totals are both negative. While possibly correct, negative spending isn't intuitive to users.
llienfec commented 6 years ago

I talked to a data user about this chart in depth today. Looking at the stats for 2016, it seems impossible that almost all of the money raised in the election happened between Dec 2017 and Jan 2018. On a gut level, the data feels wrong.

I need to keep digging into this data, but may run out of time before the sprint is over. One theory is a problem with the API data getting pulled into the graph. I took a snapshot of the current graph to see if the data points decrease once the new filings come in. It's going to take some time and some conversations about how the data is getting pulled in to figure out what's happening.

image

jwchumley commented 6 years ago

I looked at the data coming out of the API and at the code that produces it. I'm pretty sure it's an API issue. For example, I pulled party disbursements by month from FECP and compared to the same thing in the API:

Month FECP Party Disb tot. Chart party Disb total
Jan 2017 $35,104,175.20 13128.39
Feb 2017 $37,702,235.49 13128.39
Mar 2017 $52,487,664.44 14768.88
Apr 2017 $42,439,368.46 14768.88
May 2017 $44,335,616.95 14768.88
Jun 2017 $47,982,589.05 16122.13
Jul 2017 $29,659,962.80 16122.13
Aug 2017 $32,569,986.13 16122.13
Sep 2017 $36,975,474.27 16122.13
Oct 2017 $32,693,446.54 16122.13
Nov 2017 $34,427,362.00 16122.13
Dec 2017 $43,231,075.39 408140.33
Jan 2018 $36,621,577.28 408140.33
Feb 2018 $43,896,441.40 408140.33
Mar 2018 $51,134,588.27 4174604.88
Apr 2018 $42,481,693.58 507156812.1
llienfec commented 6 years ago

Another user issue:

image

patphongs commented 6 years ago

Back up plan is to use this static data line chart PR https://github.com/fecgov/fec-cms/pull/2044. This is a band-aid until we can fix the /totals/by_entity endpoint.

patphongs commented 6 years ago

@jwchumley Has another database method to try and handle the static data in this API endpoint. Therefore, I am closing my static data line chart PR #2044 for now and can resurrect if we find that we still need it. Thanks!

jwchumley commented 6 years ago

I've created a script that makes correct data rows to replace the source for the entity_chart_mv. My rows are cumulative from day 1 of a cycle to each month of the cycle. Joseph is adding rows for each month (day 1 of each moth to the last day of each month) to be consistent with the existing view. Once that is done we can replicate to PG so Helen can rework the endpoint to use the new data source.

AmyKort commented 6 years ago

Should we close this research ticket in favor of implementation tickets?