Investigate and document tests - Workload/Scalability

[verdx]

Parent issue	https://github.com/wazuh/wazuh-qa/issues/4241
Pipeline state	:red_circle:
Tests state	:red_circle:
Jenkins link	https://ci.wazuh.info/job/CLUSTER-Workload_benchmarks_metrics/
Branch used	4.4

Description

This issue aims to analyze and document all about the Workload and Scalability tests. Jenkins parameters, pytest location, steps in the test and current state of it.

What is being tested?

This pipeline tests Wazuh's response to workload and it's scalability. This is done by creating a cluster and simulating a given amount of agent and then sending events to the simulated agents sockets. From the agent's and cluster response data CSV files are generated and used to create graphs that show response CPU usage of wazuh-apid and wazuh-clusterd daemons.

State

• The expected graphs are not being generated, this is reported in Issue https://github.com/wazuh/wazuh-jenkins/issues/4748.

  • Execution Evidence: CLUSTER-Workload_benchmarks_metrics_279_ARTIFACTS

• The pipeline also runs the Performance tests from wazuh_qa repository. Research on this tests was handled in https://github.com/wazuh/wazuh-qa/issues/4260

• Currently, tests are centered in wazuh-apid and wazuh-clusterd daemons, and since EPS can be sent to different sockets, they do not affect the tests. They could be used to test other daemons like analysisd or remoted

Parameters

• Base general parameters
  • VERBOSITY
  • JENKINS_REFERENCE
  • DESTROY_INSTANCES
• Base parameters for all tests:
  • TARGET_PKG_VERSION
  • TARGET_PKG_REVISION
  • TARGET_REPOSITORY: package location
  • MANAGERS_AWS_DEPLOYMENT: EC2 or ECS
  • MANAGER_INSTANCES: Type of AWS instance used for Managers
  • AGENT_INSTANCES: Type of AWS instance used for Agents
  • DEVELOPMENT Set an ECR image name and use the CUSTOM_IMAGE parameter to use an alternative ECR image.
    • CUSTOM_IMAGE
  • CUSTOM_TASK_DEFINITION_API: Set an ECS Task Definition name to use an alternative ECS Task Definition for Wazuh API.
  • CUSTOM_TASK_DEFINITION_SIMULATE: Set an ECS Task Definition name to use an alternative ECS Task Definition for the Simulated agents.
  • CUSTOM_TASK_DEFINITION_VISUALIZATION: Set an ECS Task Definition name to use an alternative ECS Task Definition for Visualizations.
• Base parameters for QA tests:
  • QA_REFERENCE: wazuh-qa branch
  • LOCAL_INTERNAL_OPTIONS_CONFIG
• Specific parameters:
  • TEST_TIME: Time that tests will run (in seconds)
  • LOAD_BALANCER: elb or wazuh_proxy
  • PROTOCOL: TCP or UDP
  • OS: Manager OS (default Debian10)
  • AGENT_SIMULATED_VERSION:
  • LABELS: Labels to identify agents. Dictionary containing labels that will be added to agents. Used as part of the keep-alive message of agents to identify each one.
  • KEEP_ALIVE_INTERVAL: Interval for agent keepalive
  • EVENTS_SIZE: default 0
  • MODE: type of events that will be checked. Agents Events, Events per Second through the API or a Hybrid of both types. Values: AGENTS, EPS, HYBRID
  • AGENTS_NUMBER: Cluster tests require 50000 agents
  • MANAGER_WORKERS: Cluster tests require 20 or 25 workers.
  • PARSE_CLUSTER_LOGS (checkbox): Extract cluster statistics and create their corresponding charts
  • CLUSTER_TESTS: Type of tests to launch (performance, reliability)
  • TEST_PERFORMANCE_API_ENDPOINTS: Launch wazuh-qa/tests/test_performance/test_api tests
  • KIBANA_API_REQUESTS: Adittional Kibana API requests
  • EXTRA_LOAD_API_REQUESTS: Additional Wazuh API requests
  • API_REQUESTS_PERCENTAGE: Percentage of extra Load API requests
  • QUEUES: EPS to be sent by each simulated agent through a given queue
  • MANAGER_MONITORING_PROCESSES: Manager process to monitor
  • MANAGER_MONITORING_STATS: Stats to monitor (Options: remote, analysis, logcollector)
  • LOGGING: log file to monitor (alerts.log, archives.log, archives.json)
  • GROUPS_NUMBER: Amount of agent groups to create and assign to agents.
  • FILES_NUMBER_PER_GROUP: Amount of Files that will be created and sent to each group.
  • BYTES_NUMBER_PER_FILE: Byte size of each file created in each group.

[mauromalara]

Review comment No.1

The issue has no ETA and Start date ⚠️

What is tested

I would change the title to “What is being tested?”

Parameters

CUSTOM_TASK_DEFINITION_SIMULATE

Explain this parameter and its difference with CUSTOM_TASK_DEFINITION_API

CUSTOM_TASK_DEFINITION_VISUALIZATION

IDEM above.

LABELS

Do they have a specific format? (Comma-separated, separated by a newline, YAML, etc.) How can I use them?

MODE

I would add more detail to this. Maybe detailing the different types would help to understand better.

QUEUES

Change the description to: “EPS to be sent by each simulated agent through a given queue”

GROUPS_NUMBER, FILES_NUMBER_PER_GROUP, BYTES_NUMBER_PER_FILE

Describe these parameters

State

There is no execution attached. Please, attach some information about this or a screenshot.

[Deblintrake09]

Applied requested changes.

[mauromalara]

Review comment No.2

Good job!!

• Remove the "End date" from the project, this field should be filled when the issue is closed.
• Complete ETA and ETA cycles, they cannot be empty
• There is no execution evidence attached (artifacts file or screenshot).

[mauromalara]

LGTM!

[davidjiglesias]

LGTM!