datamade / bga-payroll

💰 How much do your public officials make?
4 stars 4 forks source link

Lean into django-pgviews for efficiency #510

Open hancush opened 3 years ago

hancush commented 3 years ago

Calculating total pay on the fly over large querysets is really inefficient. In #508 and #509, we added a materialized view containing the calculated total pay, and refactored the employer percentile queries to leverage it.

There are some outstanding ideas about configuring our view so we can interact with it via the ORM in #508 that it would be good to proceed with.

We might also consider applying this refactor to the person page, which calculates percentiles via model properties with queries that calculate total pay on the fly. (Not my finest idea, I think...)