slanatech / swagger-stats

API Observability. Trace API calls and Monitor API performance, health and usage statistics in Node.js Microservices.
https://swaggerstats.io/
MIT License
891 stars 136 forks source link

API Responses page in built-in Telemetry UI is not responding with ~1000 endpoints #139

Closed hieuxlu closed 3 years ago

hieuxlu commented 3 years ago

My web application has around ~80~ 1000 endpoints, so the API Responses page is unresponsive because of too many charts, making the page unusable.

What would be the best way to address this issue in Swagger Stats UI. Something that comes in my mind:

I would like to discuss if we want to fix this. I am able to contribute to fixing this with a PR, once we agreed on the solution.

sv2 commented 3 years ago

Thanks ! Can you share your /stats API response with data for that page, so I can use it to reproduce ? I will try to simulate as well

hieuxlu commented 3 years ago

Sorry, I took a closer look and it turns out there were 1000 different endpoints. I was mistaken with number of express route files (~100), each of which might have ~10 endpoints. So in total there were ~1000 individual end points.

Do you think swagger-stats's built-in Telemetry UX could some how still works in this cases, or could we some how improve API Responses page for such use-cases?

I created this minimal repo with ~1000 generated endpoints to reproduce https://github.com/hieuxlu/swagget-stats-1000-endpoints

sv2 commented 3 years ago

Thank you! Will review

hieuxlu commented 3 years ago

@sv2 Is there any update from your end? I could help creating a PR if we agree on a solution. I personally perfer simple pagination (.e.g <= 20,50,100 APIs/page) to reduce the cost of chart rendering first. Does that sound good to you?

sv2 commented 3 years ago

Good progress - now looks much more responsive with 1000 api endpoints: https://github.com/slanatech/swagger-stats-ux/commit/3044258f33a0acda87e3c87f66fd962b644c94d5

Perhaps you can try building with latest UX package to validate from your side ? will release this update shortly

hieuxlu commented 3 years ago

@sv2 Nothing seems to change on my side. I built the new ux, replaced the old one in node_modules/swagger-stats/ux, double checked to verify it's using the new file hashes, which it did. However, the page is still not responding for ~5 minutes.

I would not blame the code or the library. There's simply too many charts. And no one would look at 1000 charts at the same time, so why do we bother rendering all of them?

sv2 commented 3 years ago

Right, the change is exactly that - to render only charts that are currently visible. Let me do a bit more testing.

sv2 commented 3 years ago

Made several corrections - please try with swagger-stats-ux 0.95.29 https://www.npmjs.com/package/swagger-stats-ux/v/0.95.29

hieuxlu commented 3 years ago

Thank you, the loading placeholder solved the issue, although we'll need to update ux directory in swagger-stats npm package as well. Will close the issue now.

sv2 commented 3 years ago

Great! Will release swagger-stats version with updated ux shortly

sv2 commented 3 years ago

swagger-stats v0.99.2 with this fix released