Emulating end-user experience in different areas of Observability on clusters which are placed under significant load and evaluating whole-stack performance between different types of deployments. Most tests are designed to put together as many "heavy" areas (such as pages with multiple visualizations) as possible and navigate the user through several pages of a particular Kibana section, measuring the scenario duration.
Install Playwright.
Examine available test scenarios and required datasets. Have ideas for new user journeys? Check the guide for creating a new test.
Create .env file in the root directory with the following environmental variables:
KIBANA_HOST =
ELASTICSEARCH_HOST =
KIBANA_USERNAME =
KIBANA_PASSWORD =
TIME_UNIT = 'Minutes'
TIME_VALUE = 15
DATE_PICKER = 'Last 24 hours'
API_KEY = 'ApiKey ...'
REPORT_FILE = ../test-results/results.json
To enable verbose logging, set the DEBUG
environmental variable:
DEBUG = "pw:api"
In case there is a need to assess the impact of alerting rules execution on performance, it is recommended to create alerting rules before generating any data by running the following command:
npx playwright test alerting_rules.api.spec.ts --project api
Note: API key is required.
By default, tests run in headless mode. To launch browsers in headed mode, use the --headed
flag.
It is recommended to run tests during ongoing data collection.
Specify project name (stateful
or serverless
) in the test command. Example:
npx playwright test --project serverless --headed
Specify test name in the test command. Example:
npx playwright test apm.serverless.spec.ts --project serverless --headed
Execute serverless.auth.ts
or stateful.auth.ts
as follows:
npx playwright test serverless.auth.ts
To authorize access to Elasticsearch resources, pass your API key into API_KEY
environmental variable. To run a suite of API tests, specify correspondent project name (api
) in the following command:
npx playwright test --project api
Playwright spits out JSON test reports that have nested structure, which not quite suitable for Elasticsearch - results for each test is a separate array with its own fields. The problem is nested field type is not supported in Kibana visualizations. To solve this, use this script to flatten and split a report by each test:
node utils\split_json_report.ts
Resulting files stored in the same directory as the original report.