datamade / openness-project-nmid

Money Trail NM - New Mexico In Depth's Campaign Finance Explorer
https://moneytrailnm.com
3 stars 1 forks source link

Add 2020-2023 contribution data and import script #177

Closed hancush closed 9 months ago

hancush commented 9 months ago

Overview

This PR adds a Makefile that downloads, compresses, and stores contribution and expenditure data, and a management command to import contribution data.

The management command takes 60-90 seconds locally, but 15-30 minutes on Heroku. As it will not be run frequently in a deployment environment, I feel trading off longer run time for the legibility of the ORM, rather than writing raw SQL, is more than worth it.

Notes

Some outstanding issues:

Testing Instructions

Tested extensively locally and against the review app.