[Vets-API] Perform load testing in EKS dev

department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:

https://depo-platform-documentation.scrollhelp.site/index.html

283 stars 203 forks source link

[Vets-API] Perform load testing in EKS dev #35363

Open jhouse-solvd opened 2 years ago

jhouse-solvd commented 2 years ago

Description

Perform load testing of the Vets-API application to ensure that it is stable and performant in the EKS dev cluster.

Technical notes

The Infrastructure team is responsible for testing, monitoring, and scaling cluster infrastructure and components
The Console Services team should devise a testing plan/methodology
The load test should be equivalent to real-world production traffic volumes

Tasks

[ ] Verify network flow and server/service metrics are within reason
[ ] Inform Vets API maintainers when the service is ready for them to load test

Acceptance Criteria

[ ] Load test has been performed against dev-api.va.gov and results recorded
[ ] Create follow-on tickets for any issues found

jhouse-solvd commented 2 years ago

The Infrastructure Team can load test the performance of EKS clusters and associated components (ie load balancers, ingress, Traefik, etc). However, application load testing will ideally be performed by the Console Services team.

@npeterson54 & @ericboehs - Let us know your thoughts on this. And, let me know if you can take on this task, or if there is something similar on your roadmap already. Thank you both!

cc: @ph-One @mchelen-gov

jhouse-solvd commented 2 years ago

Eric and I discussed this today. We think it's wise to engage the Infrastructure Team's product manager and engineering leadership to help make a migration and testing plan for Q2.

ph-One commented 2 years ago

This is for tracking purposes, but load testing should be done by the owner of the application/service

jhouse-solvd commented 2 years ago

This is still worthwhile to explore, though I think it might need to be broken into separate tickets with more clear acceptance criteria. I'm detaching this from the initiative to 'make vets-api operational in dev' as I don't believe this is a blocker to that initiative.