search

fecgov / openFEC

The first RESTful API for the Federal Election Commission. We're aiming to make campaign finance more accessible for journalists, academics, developers, and other transparency seekers.
https://api.open.fec.gov/developers
Other
480 stars 106 forks source link

Add data to API #1434

Closed LindsayYoung closed 8 years ago

LindsayYoung commented 8 years ago

Talked to @paulclark2 and there is some additional information that needs to be created or transferred from the FEC. Paul, for your connivence, you can check the boxes and add comments to this thread so we know when you add the resources. Thanks!

Additional data

Views and Tables (lower priority)

This came from discussion here: https://docs.google.com/document/d/1dZpMv8K6hikRCHnXUSiEt0hQ4twRQNBeh3bZD786yJs/edit# (The document is meant to be temporary, the todos are GitHub issues, and the explanations will go into the API and site documentation.)

I am modifying this list as needs change.

noahmanger commented 8 years ago

Any updates on this before sprint planning?

LindsayYoung commented 8 years ago

Paul started this but it is not in production yet

PaulClark2 commented 8 years ago

just tested the EC view change to confirm I'm not breaking the data catalog. It looks good to me. I'll ask our DBA team to move it to production on Monday.

The new columns are: receipt_dt (Form 9) ELECTION_TP (Form 94 - the candidate list) FILE_NUM (Form 9) AMNDT_IND (Form 9)

jwchumley commented 8 years ago

Here is a list of Line numbers and quasi English descriptions as a starting point for line number filter labels.
SUMMARY_FORMAT_DISPLAY.txt

PaulClark2 commented 8 years ago

and via email (I'm not sure Jean or Roahn have GitHub accounts)

Josh, Lindsay, Jean and Rohan,

I'm not sure what's the best way to coordinate this change. I'll let you all tell me.

-Paul

Communication Costs view:
I've added one (1) column and change the data type of one (1) column.

New column F7_RECEIPT_DT, Data type change - NUMBER TO DATE TRANSACTION_DT

Electioneering Communication view I've added four (4) columns.
RECEIPT_DT, ELECTION_TP, FILE_NUM, AMNDT_IND

COMMUNICATION_COSTS_VW (this cause the loading program fail since the type not match) ERROR: cannot alter type of a column used by a view or rule DETAIL: rule _RETURN on materialized view ofec_communication_cost_mv depends on column "transaction_dt"

ELECTIONEERING_COM_VW (this will cause the column order in postGre database and in FECP be different. However, the loading program will still work) ERROR: cannot drop table electioneering_com_vw column pg_date because other objects depend on it DETAIL: materialized view ofec_electioneering_mv depends on table electioneering_com_vw column pg_date

noahmanger commented 8 years ago

@LindsayYoung can you update the checklist above to reflect what we have / still need? (Sorry, having a hard time deciphering).

PaulClark2 commented 8 years ago

these should be checked off

Add a column on electioneering_com_vw for receipt_date Add a column on communication_costs_vw for receipt_date Add a column on communication_costs_vw electioneering_com_vw F94.ELECTION_TP, (election type from the F94) Add a column on communication_costs_vw electioneering_com_vw F9.FILE_NUM, (file number from the form 9) Add a column on communication_costs_vw electioneering_com_vw F9.AMNDT_IND (amendment indicator from the form)

noahmanger commented 8 years ago

Is this issue still relevant after our new plan? Or are these tables not included in that?

PaulClark2 commented 8 years ago

Clean replacement for dimparty & clean replacement for dimreport will be taken care of by the new data plan.

A table/view that crosswalks the RAD and reporting election codes: TRC_ELECTION_TYPE should take care of RAD report codes. Disclosure report codes are in REF_RPT_TP. You have these two tables. We also have a disclousre table REF_ELECTION_TP. You do not have this table.

A table/view that has the schedule A line numbers with a text explanation. This is probably still needed.

Add a column on filings that works as a true amendment indicator. For F1 and F2 we use a view of the filings to do this.

noahmanger commented 8 years ago

Great, thanks. I updated the initial comment and pulled those two tasks into their own issues and assigned them to @PaulClark2 so they're easier to track:

This issue is referenced in those, so closing.