Our current codebase on stats endpoints run all time computations as and when an API call is made and all these operations are performance heavy on RDS. I propose to pick some best practices from Missing Maps Leaderboard backend.
For Leaderboard, all the stats computations are done prior and stored in a separate result set - it could be views or tables. We could use tables for TM.
And the API directly filters out rows from the result set
So we save loads of time on task history read operations for stats endpoints. Pre computing data will also help us include additional fields on our endpoints without worrying about DB performance.
We're already following a similar approach for notifications, let's also work on similar lines for stats.
Create two separate table for stats on projects and users - project_stats and user_stats
Use a trigger function to insert/update entries in these tables as and when a new entry is done to task history
This will be a heavy lift and we can begin here after wrapping up notifications.
ramyaragupathy
commented
4 years ago
Our current codebase on stats endpoints run all time computations as and when an API call is made and all these operations are performance heavy on RDS. I propose to pick some best practices from Missing Maps Leaderboard backend.
So we save loads of time on task history read operations for stats endpoints. Pre computing data will also help us include additional fields on our endpoints without worrying about DB performance.
We're already following a similar approach for notifications, let's also work on similar lines for stats.
project_statsanduser_statsThis will be a heavy lift and we can begin here after wrapping up notifications.
cc @xamanu @willemarcel @fitoria @JorgeMartinezG