wazuh / wazuh

Wazuh - The Open Source Security Platform. Unified XDR and SIEM protection for endpoints and cloud workloads.
https://wazuh.com/
Other
11.03k stars 1.67k forks source link

Release 4.7.1 - RC 1 - System tests #20478

Closed pro-akim closed 11 months ago

pro-akim commented 11 months ago

The following issue aims to run all system tests for the current release candidate, report the results, and open new issues for any encountered errors.

System tests information

Main release candidate issue https://github.com/wazuh/wazuh/issues/20463
Version 4.7.1
Release candidate # Release Candidate 1
Tag https://github.com/wazuh/wazuh-qa/tree/v4.7.1-rc1
Previous system tests issue https://github.com/wazuh/wazuh/issues/20224

Instructions

To run tests in an AWS EC2 virtual environment, the following requirements will need to be met:

Environment EC2
Basic_cluster Ubuntu 22.04.2 LTS C5.XLarge 15GB SSD
Big_cluster_40_agents Ubuntu 22.04.2 LTS T3.Large 60GB SSD
Agentless_cluster Ubuntu 22.04.2 LTS T3.Large 30GB SSD
Four_manager_disconnected_node Ubuntu 22.04.2 LTS T3.Large 30GB SSD
One_manager_agent Ubuntu 22.04.2 LTS T3.Large 30GB SSD
Manager_agent Ubuntu 22.04.2 LTS T3.Large 30GB SSD
Enrollment_cluster Ubuntu 22.04.2 LTS T3.Large 30GB SSD
Basic_environment Ubuntu 22.04.2 LTS T3.Large 30GB SSD

These requirements should be requested to the @wazuh/cicd team via https://github.com/wazuh/internal-devel-requests.

For further information, check https://github.com/wazuh/wazuh-qa/tree/master/tests/system/README.md

Test report procedure

All individual test checks must be marked as:
Pass The test ran successfully.
Xfail The test was expected to fail and it failed. It must be properly justified and reported in an issue.
Skip The test was not run. It must be properly justified and reported in an issue.
Fail The test failed. A new issue must be opened to evaluate and address the problem.
All test results must have one the following statuses:
:green_circle: All checks passed.
:red_circle: There is at least one failed check.
:yellow_circle: There is at least one expected fail or skipped test and no failures.

Any failing test must be properly addressed with a new issue, detailing the error and the possible cause. It must be included in the Fixes section of the current release candidate main issue.

Any expected fail or skipped test must have an issue justifying the reason. All auditors must validate the justification for an expected fail or skipped test.

An extended report of the test results must be attached as a zip or txt. This report can be used by the auditors to dig deeper into any possible failures and details.

Auditors validation

The definition of done for this one is the validation of the conclusions and the test results from all auditors.

All checks from below must be accepted in order to close this issue.

Conclusion :red_circle:

New issue

pro-akim commented 11 months ago

Test results

Reports
[Report_agentless.zip](https://github.com/wazuh/wazuh/files/13488777/Report_agentless.zip) [Report_basic_environment.zip](https://github.com/wazuh/wazuh/files/13490003/Report_basic_environment.zip) [Report_enrollment_cluster.zip](https://github.com/wazuh/wazuh/files/13491532/Report_enrollment_cluster.zip) [Report_big40.zip](https://github.com/wazuh/wazuh/files/13498396/Report_big40.zip) [Report_four_manager.zip](https://github.com/wazuh/wazuh/files/13498397/Report_four_manager.zip) [Report_manager_agent.zip](https://github.com/wazuh/wazuh/files/13498714/Report_manager_agent.zip) [Report_one_manager.zip](https://github.com/wazuh/wazuh/files/13499039/Report_one_manager.zip) [Report_basic_cluster.zip](https://github.com/wazuh/wazuh/files/13500476/Report_basic_cluster.zip)

Test name Status Pass XFail/Skip Fail Issues Ref.
One Manager Agent
1 test_fim/test_files/test_files_cud.py :green_circle: 9 0 0
2 test_fim/test_synchronization/test_synchronization.py :green_circle: 6 0 0
Four Manager Disconnected Node
3 test_cluster/test_agent_groups/test_agent_groups_new_cluster_node.py :green_circle: 2 0 0
Agentless Cluster
4 test_cluster/test_integrity_sync/test_integrity_sync.py :green_circle: 4 0 0
5 test_cluster/test_ruleset_sync_status/test_ruleset_sync_status.py :green_circle: 1 0 0
6 test_jwt_invalidation/test_change_rbac_mode.py :green_circle: 4 0 0
7 test_jwt_invalidation/test_change_security_resources.py :green_circle: 4 0 0
8 test_jwt_invalidation/test_disconnected_nodes.py :green_circle: 1 0 0
9 test_jwt_invalidation/test_revoke_endpoint.py :green_circle: 6 0 0
10 test_jwt_invalidation/test_update_password.py :green_circle: 4 0 0
Enrollment Cluster
10 test_cluster/test_agent_enrollment/test_agent_enrollment.py :green_circle: 1 0 0
11 test_cluster/test_agent_groups/test_agent_default_group_added.py :green_circle: 3 0 0
12 test_cluster/test_agent_groups/test_assign_agent_group_with_enrollment.py :green_circle: 1 0 0
13 test_cluster/test_agent_groups/test_assign_agent_never_connected_to_group.py :green_circle: 2 0 0
14 test_cluster/test_agent_groups/test_assign_agent_to_a_group_api.py :green_circle: 4 0 0
15 test_cluster/test_agent_groups/test_assign_agent_to_a_group_by_tool.py :green_circle: 2 0 0
16 test_cluster/test_agent_groups/test_assign_agent_to_a_group.py :green_circle: 4 0 0
17 test_cluster/test_agent_groups/test_assign_groups_guess.py :green_circle: 12 0 0
18 test_cluster/test_agent_groups/test_agent_groups.py :green_circle: 1 0 0
19 test_cluster/test_agent_groups/test_remove_group.py :green_circle: 10 0 0
Manager Agent
20 test_active_response_log_format/test_active_response_log_format.py :green_circle: 3 0 0
Basic Cluster
21 test_cluster/test_agent_info_sync/test_agent_info_sync.py :green_circle: 2 0 0
22 test_cluster/test_agent_key_polling/test_agent_key_polling.py :green_circle: 1 0 0
23 test_cluster/test_agent_files_deletion/test_agent_files_deletion.py :red_circle: 1 0 0 https://github.com/wazuh/wazuh-qa/issues/4735
24 test_multigroups/test_multigroups.py :green_circle: 4 0 0
25 test_cluster/test_agent_groups/test_agent_groups_forced_change.py :green_circle: 3 0 0
Big Cluster 40 Agents
26 test_cluster/test_agent_groups/test_groups_sync_default.py :green_circle: 2 0 0
27 test_cluster/test_agent_groups/test_groups_sync_time.py :green_circle: 2 0 0
Basic Environment
28 test_agent_auth/test_agent_auth.py :green_circle: 20 0 0
29 test_enrollment/test_enrollment.py :green_circle: 20 0 0
juliamagan commented 11 months ago

LGTM

damarisg commented 11 months ago

LGTM!