The current data fetching algorithm didn't collect all the required data and had massive performance issues. This PR aims to fix both of these issues. With this PR, the table should be on-par with the data from the original Python script.
Description
This PR includes multiple improvements:
Github PR data is now sorted after UPDATED instead of CREATED, allowing us to get all PR that were worked on
Instead of always querying for fetched entities in the DB, we implement caching for improved performance (see performance test below)
GET /leaderboard now filters the timeframe for the data before calculating the response
Performance Test
#PRs
develop-branch
PR (no caching)
PR (caching)
8
90s
-
-
53
impossible
5 min
3 min
In my test runs the new fetching process now takes about 3 minutes (53 PRs), in comparison to the original 1 minute (8 PRs) and 5 minutes without caching (53 PRs).
Screenshots (if applicable)
Checklist
General
[x] PR title is clear and descriptive
[x] PR description explains the purpose and changes
[x] Code follows project coding standards
[x] Self-review of the code has been done
[x] Changes have been tested locally
Server (if applicable)
[x] Code is performant and follows best practices
[x] No security vulnerabilities introduced
[x] Proper error handling has been implemented
[x] Added tests for new functionality
[x] Changes have been tested in different environments (if applicable)
Motivation
The current data fetching algorithm didn't collect all the required data and had massive performance issues. This PR aims to fix both of these issues. With this PR, the table should be on-par with the data from the original Python script.
Description
This PR includes multiple improvements:
GET /leaderboard
now filters the timeframe for the data before calculating the responsePerformance Test
develop
-branchIn my test runs the new fetching process now takes about 3 minutes (53 PRs), in comparison to the original 1 minute (8 PRs) and 5 minutes without caching (53 PRs).
Screenshots (if applicable)
Checklist
General
Server (if applicable)