End to end and stress tests

arschles commented 3 years ago

We currently have mostly unit tests inside this repository, along with a few integration tests (like one for the interceptor coming in #206). I propose that we add more testing strategies to this project, including:

end to end: a test that installs the addon to a Kubernetes cluster along with KEDA and any other dependencies, installs an app and an HTTPScaledObject, and runs a quick suite of requests to trigger scale-from-zero, scale up, and scale back down to zero. this test should check that the app and the interceptor both scale properly, and that the external scaler, operator, and interceptor's admin endpoints all report the expected things (NOTE: KEDA does this, and there is a prototype of something similar in #206)
stress: a test that runs a large number of requests concurrently against an interceptor fleet, optionally for a large period of time. also possibly inspect internal admin endpoints along the way to ensure that metrics report as expected. the main purpose of this test is to ensure that, from the outside, requests get serviced as expected

arschles commented 3 years ago

@ajanth97 are you interested in implementing either one of these at some point?

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

ajanth97 commented 2 years ago

@arschles sorry I just saw this. yeah will do when I get some time

arschles commented 2 years ago

sounds good @ajanth97 - thanks!

v-shenoy commented 2 years ago

I am currently working on this.

JorTurFer commented 1 year ago

e2e test are executed on every PR/Push https://github.com/kedacore/http-add-on/pull/564

HongboFei commented 1 year ago

should we close this one if done?

JorTurFer commented 1 year ago

Stress tests are still pending, we could cose this and open another issue for stress test if we want to have it more accuracy tracked

HongboFei commented 1 year ago

OK, thanks Jorge. That;s OK. We can keep it what it is.

JorTurFer commented 1 year ago

@MrsDaehin , may you be interested in help with this too? (after core load tests are ready, ofc) This is easier to test indeed because this is closer to an API (and here the load should be higher)

MrsDaehin commented 1 year ago

we can discuss this. It will be easy once we got k6 running.

Leonardo-Ferreira commented 5 months ago

@arschles, since you proposed the definition of a stress test, shouldn't we stipulate a criteria for "acceptance"? like, "given that every request will take 1sec at the backend, for every pod of the http interceptor with 0.1 cpu and 128mb, the stress test will fail if: the interceptor crashes, the interceptor gets OOM killed, less than X requests are handled in parallel at any given time, less than Y requests are handled over the period of 10min" and so on?

kedacore / http-add-on

End to end and stress tests #238