Is your feature request related to a problem? Please describe.
Currently unit, integration and functional tests are triggered during the CI build on Azure DevOps.
Impact of new features is not monitored automatically. It would be very beneficial, to setup some performance benchmark as a part of the pipeline that will run simple performance check and compare the results agains some threshold.
Describe the solution you'd like
Performance benchmark is a part of the build pipeline.
The very basic idea would be to run the knotx-starterkit as a docker container and use some load generator (this looks quite promising) and gather results.
Additionally there should be threshold (KPI) set that will fail the build if KPI is not met.
Describe alternatives you've considered
There are a few alternatives:
instead of knoxt-starterkit use some knotx-examples to cover more complex scenarios in tests
performance checks are scheduled nightly not as parto fo the build pipeline
Additional context
As part of this task, some Azure DevOps RnD should be done, to check if that's possible to run such performance benchmark during the pipeline and to check if any additional servers to run Knot.x are required or not.
Is your feature request related to a problem? Please describe. Currently unit, integration and functional tests are triggered during the CI build on Azure DevOps. Impact of new features is not monitored automatically. It would be very beneficial, to setup some performance benchmark as a part of the pipeline that will run simple performance check and compare the results agains some threshold.
Describe the solution you'd like Performance benchmark is a part of the build pipeline. The very basic idea would be to run the knotx-starterkit as a docker container and use some load generator (this looks quite promising) and gather results. Additionally there should be threshold (KPI) set that will fail the build if KPI is not met.
Describe alternatives you've considered There are a few alternatives:
Additional context As part of this task, some Azure DevOps RnD should be done, to check if that's possible to run such performance benchmark during the pipeline and to check if any additional servers to run Knot.x are required or not.