Determine what QA is needed for the file upload project

[ewilson-adhoc]

User Story

As an IET developer, I would like to begin to determine what our testing strategy is for this project, so that I will be able to determine the performance/robustness of our prototype implementations.

Expected Behavior

This ticket is a reminder to reach out to the QA team and begin figuring out what the test process should look like, document what tools are available and what the protocol is like for something like stress testing.

Acceptance Criteria

• [x] IET has begun to think about QA

[considerable]

Platform developer docs section Testing and releasing among other things says, "Contract testing on VA.gov is accomplished using Pact" and "Load tests need to be run before launching new endpoints or substantial updates to existing endpoints to ensure the stability of the API".
Locust is the current Platform load tool that lets you define test behavior in Python code and has a web interface for interactive testing. It also supports creating a swarm of test instances.

[considerable]

One kind of load testing not done by Platform in the past is Chaos Testing, a technique that emerged along with the need to test Microservices on Kubernetes. Amazon blog: Chaos Engineering with LitmusChaos on Amazon EKS explains how Chaos experiment can be launched using the Litmus ChaosCenter UI by creating a workflow to test
Hypothesis: Amazon EKS cluster should auto scale when cluster capacity is insufficient to run the pods.

[ewilson-adhoc]

I found this page https://vfs.atlassian.net/wiki/spaces/IET/pages/2107572231/File+Upload+Test+Plan let's capture some of this data (perhaps in an RFC we can give to the QA team for approval?)

[considerable]

ok, one more thing to capture - looks like the infra team has already been looking into the Chaos testing setup - https://vfs.atlassian.net/wiki/spaces/SRE/pages/2032599080/Initial+Questions+Tasks+from+Zach+Lewis

[ewilson-adhoc]

One of the questions that we should be thinking about is what sort of testing we need to do to determine if our new services is "better" than the old service. I think a good first step might be to look at the existing endpoints and see if we can find results of previous load tests. I don't think we need to worry about chaos testing for this project (the fact that we're on EKS will help make our system self-healing).

Do you know if it is feasible to use Pact on endpoints that are still liable to change? We'll definitely want to use it once we've nailed down the inputs/outputs of our API -- but for the initial prototype, the API is liable to change.

[ewilson-adhoc]

I think we've satisfied the acceptance criteria of this. We can reopen additional tickets as we need to do QA work.