datamade / openness-project-nmid

Money Trail NM - New Mexico In Depth's Campaign Finance Explorer
https://moneytrailnm.com
3 stars 1 forks source link

Implement expenditure import #178

Closed hancush closed 8 months ago

hancush commented 9 months ago

Overview

This PR extends the contribution import to also import expenditures. It also updates the Makefile recipes to import either type from a local file or S3.

Testing Instructions

derekeder commented 9 months ago

a few observations from clicking around:

Screenshot 2023-12-07 at 3 39 05 PM

These candidates look to be for the same person:

https://openness-pro-hcg-exp-s43eps6ml.herokuapp.com/candidates/michelle-lujan-grisham-2243/ https://openness-pro-hcg-exp-s43eps6ml.herokuapp.com/candidates/michelle-lujan-grisham/

As do these:

https://openness-pro-hcg-exp-s43eps6ml.herokuapp.com/candidates/hector-balderas-3046/ https://openness-pro-hcg-exp-s43eps6ml.herokuapp.com/candidates/hector-balderas/

Screenshot 2023-12-07 at 3 55 03 PM

derekeder commented 9 months ago

Nice! The duplicates and errors I saw in my last review are resolved.

Seeing some other things that raise my eyebrows:

Home page title says 'Lobbyist Portal' when I think we are hiding that part.

Screenshot 2023-12-13 at 3 31 52 PM

Hector Balderas has campaigns after 2020, but no financial data after 2020. He also stopped being recognized as a (D): https://openness-pro-hcg-exp-s43eps6ml.herokuapp.com/candidates/hector-balderas-3046/

Screenshot 2023-12-13 at 3 30 00 PM

Of the top 10 Committees, only one has contributions after 2020. Does this mean the old ones don't exist in the new system? Or were they renamed? Is it accurate to show committees in the top 10 (based on cash-on-hand) if we don't have recent data for them?

Screenshot 2023-12-13 at 3 35 16 PM

hancush commented 8 months ago

@derekeder tl;dr - The issues you pointed out have to do with missing data or are unrelated to this PR. Can this be staged and these issues addressed separately?

Home page title says 'Lobbyist Portal' when I think we are hiding that part.

This is not relevant to these changes. Can I address this separately?

Hector Balderas has campaigns after 2020, but no financial data after 2020. He also stopped being recognized as a (D): https://openness-pro-hcg-exp-s43eps6ml.herokuapp.com/candidates/hector-balderas-3046/

This is a product of Office information missing from contribution and expenditure data, which we'll have to address separately. I've already opened a thread with Marjorie about how to handle this.

Of the top 10 Committees, only one has contributions after 2020. Does this mean the old ones don't exist in the new system? Or were they renamed? Is it accurate to show committees in the top 10 (based on cash-on-hand) if we don't have recent data for them?

I didn't import 2021 or 2022 data, to save time. This will likely change once that comes in. Otherwise, a great question for Marjorie.