search

wazuh / wazuh

Wazuh - The Open Source Security Platform. Unified XDR and SIEM protection for endpoints and cloud workloads.
https://wazuh.com/
Other
9.34k stars 1.48k forks source link

Review failed 4.9.0 API integration tests #23314

Closed GGP1 closed 4 days ago

GGP1 commented 1 week ago

Description

During the development of https://github.com/wazuh/wazuh/issues/22891 we found that there are many tests failing in the 4.9.0 branch.

All the tests except one passed when executed locally using the branch fix/22891-ait-env but we should still investigate what is causing the error in Jenkins. We've also found an error from wazuh-agentd that failed when requested to restart.

wazuh-agentd error ```console E tavern.util.exceptions.TestFailError: Test 'Try to restart a list of agents (Partially allowed, user aware)' failed: - Key mismatch: (expected["error"] = '2' (type = ), actual["error"] = '1' (type = )) - Error calling validate function '': Traceback (most recent call last): File "/home/gasti/work/wazuh/venv/lib/python3.10/site-packages/tavern/response/base.py", line 152, in _maybe_run_validate_functions vf(response) File "/home/gasti/work/wazuh/venv/lib/python3.10/site-packages/tavern/util/extfunctions.py", line 98, in inner result = func(response, *args, **kwargs) File "/home/gasti/work/wazuh/api/test/integration/tavern_utils.py", line 511, in healthcheck_agent_restart check_agentd_started(response, agents_list) File "/home/gasti/work/wazuh/api/test/integration/tavern_utils.py", line 465, in check_agentd_started raise ProcessLookupError("The wazuh-agentd daemon was not started after requesting the restart") ProcessLookupError: The wazuh-agentd daemon was not started after requesting the restart ```

Build: https://ci.wazuh.info/job/Test_integration_endpoints/4524/ Artifacts: artifacts.zip

Results | **Test name** | **Pass** | **XPass** | **Skip** | **XFail** | **Fail** | **Issues Ref.** | **Status** | |--|--|--|--|--|--|--|--| | test_active_response_endpoints.tavern.yaml | 1 | 0 | 0 | 0 | 1 | | :red_circle: | | test_agent_DELETE_endpoints.tavern.yaml | 6 | 0 | 0 | 0 | 0 | | :green_circle: | | test_agent_GET_endpoints.tavern.yaml | 95 | 0 | 0 | 0 | 0 | | :green_circle: | | test_agent_POST_endpoints.tavern.yaml | 6 | 0 | 0 | 0 | 0 | | :green_circle: | | test_agent_PUT_endpoints.tavern.yaml | 10 | 0 | 0 | 0 | 0 | | :green_circle: | | test_cdb_list_endpoints.tavern.yaml | 5 | 0 | 0 | 0 | 0 | | :green_circle: | | test_cluster_endpoints.tavern.yaml | 46 | 0 | 0 | 0 | 3 | | :red_circle: | | test_decoder_endpoints.tavern.yaml | 25 | 0 | 0 | 0 | 0 | | :green_circle: | | test_default_endpoints.tavern.yaml | 1 | 0 | 0 | 0 | 0 | | :green_circle: | | test_event_endpoints.tavern.yaml | 5 | 0 | 0 | 0 | 0 | | :green_circle: | | test_experimental_endpoints.tavern.yaml | 12 | 0 | 0 | 0 | 0 | | :green_circle: | | test_logtest_endpoints.tavern.yaml | 2 | 0 | 0 | 0 | 0 | | :green_circle: | | test_manager_endpoints.tavern.yaml | 35 | 0 | 0 | 0 | 0 | | :green_circle: | | test_mitre_endpoints.tavern.yaml | 7 | 0 | 0 | 0 | 0 | | :green_circle: | | test_overview_endpoints.tavern.yaml | 0 | 0 | 0 | 0 | 1 | | :red_circle: | | test_rbac_black_active_response_endpoints.tavern.yaml | 2 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_agent_endpoints.tavern.yaml | 41 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_cdb_list_endpoints.tavern.yaml | 5 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_cluster_endpoints.tavern.yaml | 19 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_decoder_endpoints.tavern.yaml | 6 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_event_endpoints.tavern.yaml | 1 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_experimental_endpoints.tavern.yaml | 12 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_logtest_endpoints.tavern.yaml | 2 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_manager_endpoints.tavern.yaml | 16 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_mitre_endpoints.tavern.yaml | 7 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_overview_endpoints.tavern.yaml | 0 | 0 | 0 | 0 | 1 | | :red_circle: | | test_rbac_black_rootcheck_endpoints.tavern.yaml | 4 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_rule_endpoints.tavern.yaml | 8 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_sca_endpoints.tavern.yaml | 2 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_security_endpoints.tavern.yaml | 25 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_syscheck_endpoints.tavern.yaml | 4 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_syscollector_endpoints.tavern.yaml | 9 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_black_task_endpoints.tavern.yaml | 1 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_active_response_endpoints.tavern.yaml | 2 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_agent_endpoints.tavern.yaml | 41 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_all_endpoints.tavern.yaml | 162 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_cdb_list_endpoints.tavern.yaml | 5 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_cluster_endpoints.tavern.yaml | 19 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_decoder_endpoints.tavern.yaml | 6 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_event_endpoints.tavern.yaml | 1 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_experimental_endpoints.tavern.yaml | 12 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_logtest_endpoints.tavern.yaml | 2 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_manager_endpoints.tavern.yaml | 16 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_mitre_endpoints.tavern.yaml | 7 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_overview_endpoints.tavern.yaml | 0 | 0 | 0 | 0 | 1 | | :red_circle: | | test_rbac_white_rootcheck_endpoints.tavern.yaml | 4 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_rule_endpoints.tavern.yaml | 8 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_sca_endpoints.tavern.yaml | 2 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_security_endpoints.tavern.yaml | 25 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_syscheck_endpoints.tavern.yaml | 4 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_syscollector_endpoints.tavern.yaml | 9 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rbac_white_task_endpoints.tavern.yaml | 1 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rootcheck_endpoints.tavern.yaml | 4 | 0 | 0 | 0 | 0 | | :green_circle: | | test_rule_endpoints.tavern.yaml | 15 | 0 | 0 | 0 | 0 | | :green_circle: | | test_sca_endpoints.tavern.yaml | 45 | 0 | 0 | 0 | 0 | | :green_circle: | | test_security_DELETE_endpoints.tavern.yaml | 15 | 0 | 0 | 0 | 0 | | :green_circle: | | test_security_GET_endpoints.tavern.yaml | 11 | 0 | 0 | 0 | 0 | | :green_circle: | | test_security_POST_endpoints.tavern.yaml | 8 | 0 | 0 | 0 | 0 | | :green_circle: | | test_security_PUT_endpoints.tavern.yaml | 9 | 0 | 0 | 0 | 0 | | :green_circle: | | test_syscheck_endpoints.tavern.yaml | 34 | 0 | 0 | 0 | 0 | | :green_circle: | | test_syscollector_endpoints.tavern.yaml | 159 | 0 | 0 | 0 | 0 | | :green_circle: | | test_task_endpoints.tavern.yaml | 2 | 0 | 0 | 0 | 0 | | :green_circle: |

Checks

The following elements have been updated or reviewed (should also be checked if no modification is required):

GGP1 commented 6 days ago

Update

I was able to reproduce the errors locally and I could find the reason behind them.

The cluster tests failed because the changes applied in https://github.com/wazuh/wazuh/issues/23195 aren't yet in 4.9.0.

Similarly, the overview endpoints initially failed because the changes from https://github.com/wazuh/wazuh/issues/22744 weren't in 4.9.0 yet, but they then failed along with the active-response one because the tests were started as soon as the containers were healthy, even though the agents shared configuration wasn't synced.

The tests didn't fail previously because the conftest contained many unnecessary sleeps that were indirectly giving time for the configuration to be synced. Since https://github.com/wazuh/wazuh/issues/22891, these sleeps were removed and the configuration often synced in time but other times it did not, resulting in sporadic errors.

To solve this, I incremented the time waited on the agent healtcheck scripts to give time for the configuration to sync.