tom-draper
commented
7 months ago Dashboard/server performance is definitely an ongoing issue. Having said that, I've never seen it take longer than 60 seconds with ~1 million requests logged, so something sounds wrong with 10 minutes. I would avoid refreshing the dashboard repeatedly as that can increase load times further.
The bulk of the delay is with the database read, rather than processing the results, even though the table is indexed as well as possible. I'm always looking for ways to improve performance, only loading partial data may have to be the way forward, but it would only pass the long loading times on to the moment you change the time period within the dashboard. It could possibly continue to load more requests in the background. It doesn't help that the server is underpowered but it's the only way to keep the service free.
JasonLovesDoggo
I think it is because the row size is too large. My web application gets about 150k requests, now approximately 300k for 2 days. It took 10 minutes for me to get data from the api.
And I found that the api directly return all these requests data, which may be reason for the front-end dashboard to load so slowly. As it has to analyzed 30k rows data.
Maybe summarize the traffic data hour by hour, only count the total count of every endpoint to another database table might be a more wise idea to send the preprocessed and simplified data to dashboard and api? Hoping for your reply!😊