Use repeated API calls using axios to get complete data i. e. Beyond 100 records from the API.
To avoid the rate-limit margin, the getDetails function would return only a maximum of 3 pages by default. That can be overridden by passing true to the second argument of getDetails. This would be the job of the person who designs the visualizations to decide.
...
// Get the data object for a user
const getDetails = async (username, getAll = false) => {
...
Why only 3 pages by default? Assuming each of the 5 endpoints have more than 3 pages to send. Since each page has 100 records, 5 × 3 × 100 = 1500 records will have to be fetched and parsed, which would easily cross the rate-limiting and would be resource-intensive.
Additional Info
The schema of returned data has not been modified.
The API service still uses simultaneous API calls by using Promises instead of a for loop to iterate through each endpoint.
It is around 3 times faster, and the performance remains the same for endpoints that don't require pagination i. e. That contain only one page.
...
// Build data array
resultArray = await Promise.all(resultArray.map(async ([ep, arr]) => {
let newData;
...
Checklist
[ ] Tests
[ ] Translations
[ ] Documentation
Screenshots
The interface remains the same and hasn't been affected in any manner.
Related Issue
Proposed Changes
axiosto get complete data i. e. Beyond 100 records from the API.getDetailsfunction would return only a maximum of 3 pages by default. That can be overridden by passingtrueto the second argument ofgetDetails. This would be the job of the person who designs the visualizations to decide.Why only 3 pages by default? Assuming each of the 5 endpoints have more than 3 pages to send. Since each page has 100 records, 5 × 3 × 100 = 1500 records will have to be fetched and parsed, which would easily cross the rate-limiting and would be resource-intensive.
Additional Info
Checklist
Screenshots
The interface remains the same and hasn't been affected in any manner.