Closed fdalmaup closed 11 months ago
aradkdj
commented
1 year ago Checked /var/ossec/framework/python/bin to confirm python3 is there
Reviewed the usages of framework/python/bin/python3.9 vs framework/python/bin/python3. The former seems to be a minority
Based on the above, the change should be safe and straightforward.
aradkdj
commented
1 year ago According to the test/environment matrix in order to run the affected tests, the basic_cluster environment has to be built. And it built successfully
aradkdj
commented
1 year ago Updated the branch locally and recreated the cluster, since 4.4 has seen merges since the last time I worked on this. Then I tried to run test_agent_files_deletion.py but encountered the following error:
$ pytest test_agent_files_deletion
======================================================================================================================== test session starts =========================================================================================================================
platform linux -- Python 3.10.11, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /home/aradkdj/Documents/work/Wazuh/wazuh-qa/tests/system, configfile: pytest.ini
plugins: testinfra-7.0.0, metadata-2.0.4, html-3.1.1
collected 1 item
test_agent_files_deletion/test_agent_files_deletion.py F [100%]
============================================================================================================================== FAILURES ==============================================================================================================================
_____________________________________________________________________________________________________________________ test_agent_files_deletion ______________________________________________________________________________________________________________________
def test_agent_files_deletion():
"""Check that when an agent is deleted, all its related files in managers are also removed."""
# Clean ossec.log and cluster.log
for hosts in managers_hosts:
host_manager.clear_file(host=hosts, file_path=os.path.join(WAZUH_LOGS_PATH, 'ossec.log'))
host_manager.clear_file(host=hosts, file_path=os.path.join(WAZUH_LOGS_PATH, 'cluster.log'))
host_manager.control_service(host=hosts, service='wazuh', state='restarted')
# Get the token
master_token = host_manager.get_api_token(master_host)
# Check if the agent is connected and reporting
agent_healthcheck(master_token)
# Get the current ID and name of the agent that is reporting to worker_host.
response = host_manager.make_api_call(host=master_host, method='GET', token=master_token,
endpoint=f"/agents?select=id,name&q=manager={worker_host}")
assert response['status'] == 200, f"Failed when trying to obtain agent ID: {response}"
try:
agent_id = response['json']['data']['affected_items'][0]['id']
agent_name = response['json']['data']['affected_items'][0]['name']
except IndexError as e:
pytest.fail(f"Could not find any agent reporting to {worker_host}: {response['json']}")
# Check that expected files exist in each node before removing the agent.
for file in files:
for host in file['hosts']:
result = host_manager.run_shell(
host, f"test -e {file['path'].format(id=agent_id, name=agent_name)} && echo 'exists'"
)
assert result, f"This file should exist in {host} but could not be found: " \
f"{file['path'].format(id=agent_id, name=agent_name)}"
# Check that agent information is in the wdb socket
for host in managers_hosts:
for query in queries:
result = host_manager.run_command(host,
f"{WAZUH_PATH}/framework/python/bin/python3.9 "
f"{script_path} '{query.format(id=agent_id)}'")
assert result, f"This db query should have returned something in {host}, but it did not: {result}"
# Remove the agent
response = host_manager.make_api_call(host=master_host, method='DELETE', token=master_token,
endpoint=f"/agents?agents_list={agent_id}&status=all&older_than=0s")
> assert response['status'] == 200, f"Failed when trying to remove agent {agent_id}: {response}"
E AssertionError: Failed when trying to remove agent 003: {'changed': False, 'connection': 'close', 'content_length': '53', 'content_type': 'application/problem+json; charset=utf-8', 'date': 'Thu, 11 May 2023 16:22:09 GMT', 'elapsed': 0, 'msg': 'Status code was 401 and not [200]: HTTP Error 401: Unauthorized', 'redirected': False, 'status': 401, 'url': 'https://localhost:55000/agents?agents_list=003&status=all&older_than=0s'}
E assert 401 == 200
test_agent_files_deletion/test_agent_files_deletion.py:106: AssertionError
====================================================================================================================== short test summary info =======================================================================================================================
FAILED test_agent_files_deletion/test_agent_files_deletion.py::test_agent_files_deletion - AssertionError: Failed when trying to remove agent 003: {'changed': False, 'connection': 'close', 'content_length': '53', 'content_type': 'application/problem+json; cha...
=================================================================================================================== 1 failed in 3985.08s (1:06:25) ===================================================================================================================Currently consulting with the team to find out what could be causing it as this should not be failing in the current state.
aradkdj
commented
1 year ago The team was not able to reproduce my failures. Asked for the resolved dependencies in their environments and found some differences. After installing from their freezes the I got a different failure:
(wazuh-qa) [aradkdj@fedora test_cluster]$ pytest test_agent_files_deletion
======================================================================================================================== test session starts =========================================================================================================================
platform linux -- Python 3.10.11, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/aradkdj/Documents/work/Wazuh/wazuh-qa/tests/system, configfile: pytest.ini
plugins: testinfra-5.0.0, metadata-2.0.4, html-3.1.1
collected 1 item
test_agent_files_deletion/test_agent_files_deletion.py F [100%]
============================================================================================================================== FAILURES ==============================================================================================================================
_____________________________________________________________________________________________________________________ test_agent_files_deletion ______________________________________________________________________________________________________________________
def test_agent_files_deletion():
"""Check that when an agent is deleted, all its related files in managers are also removed."""
# Clean ossec.log and cluster.log
for hosts in managers_hosts:
host_manager.clear_file(host=hosts, file_path=os.path.join(WAZUH_LOGS_PATH, 'ossec.log'))
host_manager.clear_file(host=hosts, file_path=os.path.join(WAZUH_LOGS_PATH, 'cluster.log'))
host_manager.control_service(host=hosts, service='wazuh', state='restarted')
# Get the token
master_token = host_manager.get_api_token(master_host)
# Check if the agent is connected and reporting
> agent_healthcheck(master_token)
test_agent_files_deletion/test_agent_files_deletion.py:73:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
master_token = 'eyJhbGciOiJFUzUxMiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ3YXp1aCIsImF1ZCI6IldhenVoIEFQSSBSRVNUIiwibmJmIjoxNjgzOTA2MTc5LCJleHAi...rZQXHrOSMum3ygEfJka5y1CqHLKQhgAGOrrEGaeWfVTkWanP-zDStr7_qMKtJ-ib2XBjzlU-YuMQsjJv1k7w5Hh_kZfI4zG9Iz-3stBIv4deeuteyEo5UH'
def agent_healthcheck(master_token):
"""Check if the agent is active and reporting."""
timeout = time() + time_to_agent_reconnect
healthy = False
while not healthy:
response = host_manager.make_api_call(host=master_host, method='GET', token=master_token,
endpoint='/agents?status=active')
assert response['status'] == 200, 'Failed when trying to get the active agents'
if int(response['json']['data']['total_affected_items']) == 4:
for item in response['json']['data']['affected_items']:
if item['name'] == agent_host and item['manager'] == worker_host:
healthy = True
elif time() > timeout:
> raise TimeoutError("The agent 'wazuh-agent3' is not 'Active' yet.")
E TimeoutError: The agent 'wazuh-agent3' is not 'Active' yet.
test_agent_files_deletion/test_agent_files_deletion.py:56: TimeoutError
====================================================================================================================== short test summary info =======================================================================================================================
FAILED test_agent_files_deletion/test_agent_files_deletion.py::test_agent_files_deletion - TimeoutError: The agent 'wazuh-agent3' is not 'Active' yet.
=================================================================================================================== 1 failed in 2893.30s (0:48:13) ===================================================================================================================I will rebuild the cluster and try again
aradkdj
commented
1 year ago After reviewing my local environment with the team and arriving at the conclusion that everything looked normal, I spun up a VM to rule out OS issues. After installing the project's dependencies this is the result:
Running test_agent_files_deletion.py
aradkdj
commented
1 year ago Running test_agent_info_sync
aradkdj
commented
1 year ago Building the four_manager_disconnected_node environment.
aradkdj
commented
1 year ago Creating the big_cluster_40_agents.
aradkdj
commented
1 year ago The modifications were made and a branch with them is available at 8d65d3ac24a6bf5e9d2a7615129d587a9e2f4869. Tests from now on will be made on that branch.
aradkdj
commented
1 year ago The basic_cluster environment was successfully built.
aradkdj
commented
1 year ago test_agent_files_deletion passed.
(venv) wazuh@ubuntu2204:~/wazuh-qa/tests/system/test_cluster$ pytest test_agent_files_deletion/
============================= test session starts ==============================
platform linux -- Python 3.9.16, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/wazuh/wazuh-qa/tests/system, configfile: pytest.ini
plugins: metadata-2.0.4, testinfra-5.0.0, html-3.1.1
collected 1 item
test_agent_files_deletion/test_agent_files_deletion.py . [100%]
======================== 1 passed in 255.42s (0:04:15) =========================
(venv) wazuh@ubuntu2204:~/wazuh-qa/tests/system/test_cluster$test_agent_info_sync passed.
(venv) wazuh@ubuntu2204:~/wazuh-qa/tests/system/test_cluster$ pytest test_agent_info_sync/
============================= test session starts ==============================
platform linux -- Python 3.9.16, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/wazuh/wazuh-qa/tests/system, configfile: pytest.ini
plugins: metadata-2.0.4, testinfra-5.0.0, html-3.1.1
collected 2 items
test_agent_info_sync/test_agent_info_sync.py .. [100%]
======================== 2 passed in 433.03s (0:07:13) =========================
(venv) wazuh@ubuntu2204:~/wazuh-qa/tests/system/test_cluster$four_manager_disconnected_node environment was successfully built
aradkdj
commented
1 year ago big_cluster_40_agents environment was successfully built.
fdalmaup
commented
11 months ago Rebased the branch fix/3949-consistently-use-python3-in-system-tests to 4.5. In the process of testing the modifications: |
Test | Environment | Status |
|---|---|---|---|
| test_cluster/test_agent_info_sync | basic_cluster | ||
| test_cluster/test_agent_key_polling | basic_cluster | ||
| test_cluster/test_agent_files_deletion | basic_cluster | :green_circle: | |
| test_multigroups/test_multigroups | basic_cluster | :green_circle: | |
| test_cluster/test_agent_groups/test_agent_groups_forced_change | basic_cluster | :green_circle: | |
| test_cluster/test_agent_groups/test_agent_groups_new_cluster_node | four_manager_disconnected_node | ||
| test_cluster/test_agent_groups/test_groups_sync_default | big_cluster_40_agents | ||
| test_cluster/test_agent_groups/test_groups_sync_time | big_cluster_40_agents |
fdalmaup
commented
11 months ago All the tests passed:
| Test | Environment | Status |
|---|---|---|
| test_cluster/test_agent_info_sync | basic_cluster | :green_circle: |
| test_cluster/test_agent_key_polling | basic_cluster | :green_circle: |
| test_cluster/test_agent_files_deletion | basic_cluster | :green_circle: |
| test_multigroups/test_multigroups | basic_cluster | :green_circle: |
| test_cluster/test_agent_groups/test_agent_groups_forced_change | basic_cluster | :green_circle: |
| test_cluster/test_agent_groups/test_agent_groups_new_cluster_node | four_manager_disconnected_node | :green_circle: |
| test_cluster/test_agent_groups/test_groups_sync_default | big_cluster_40_agents | :green_circle: |
| test_cluster/test_agent_groups/test_groups_sync_time | big_cluster_40_agents | :green_circle: |
Description
During https://github.com/wazuh/wazuh/issues/13373, when running the system tests, there were found several lines referring to Wazuh's embedded Python interpreter using
python3.9both in the provisioning tasks of the environments needed (basic_cluster,big_cluster_40_agentsandfour_manager_disconnected_node) and the tests themselves. For example: https://github.com/wazuh/wazuh-qa/blob/db4775bac707288f0a2aafbea99ae4f58029f281/tests/system/test_cluster/test_agent_info_sync/test_agent_info_sync.py#L109-L112This should be modified in order to use
python3as the default line to execute the interpreter and make the system tests independent of the interpreter that is being used in the Wazuh branch to be tested.Tasks
basic_cluster.big_cluster_40_agents.four_manager_disconnected_node.test_agent_files_deletion.py.test_agent_info_sync.py.test_agent_files_deletion.py.test_agent_info_sync.py.basic_cluster.big_cluster_40_agents.four_manager_disconnected_node.