rauldpm commented 4 months ago

Description

We need to update the DTT1 development README files to state the chosen workflow tool name: JobFlow

The name should be reflected in the document title:

https://github.com/wazuh/wazuh-qa/blob/4495-dtt1-release/deployability/modules/workflow_engine/README.MD
- JobFlow - Orchestrator

Then, modify all Workflow engine references to: JobFlow (note: modify only documentation, not command lines)

Tasks

[ ] Add JobFlow to README files
[ ] Change workflow_engine references to jobflow (code and directories)

mhamra commented 4 months ago

UPDATE

I updated the README.md files with the new name.

mhamra commented 4 months ago

UPDATE

I've renamed the workflow_engine folder and all the references in the source code and the documentation.

I've installed the jobflow package in a new environment and run the following example:

(dtt1) marcelo@marcelo-B460-AORUS-PRO-AC:~/wazuh/wazuh-qa/deployability$ python -m jobflow modules/jobflow/examples/agent/vagrant/test-agent-complete-1.yaml 
[2024-06-07 15:45:29] [INFO] [10222] [MainThread] [jobflow]: Executing DAG tasks.
[2024-06-07 15:45:29] [INFO] [10222] [MainThread] [jobflow]: Executing tasks in parallel.
[2024-06-07 15:45:29] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-manager-linux-ubuntu-22.04-amd64] Starting task.
[2024-06-07 15:46:17] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-manager-linux-ubuntu-22.04-amd64] Finished task in 47.67 seconds.
[2024-06-07 15:46:17] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [provision-manager-linux-ubuntu-22.04-amd64] Starting task.
[2024-06-07 15:54:46] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [provision-manager-linux-ubuntu-22.04-amd64] Finished task in 509.35 seconds.
[2024-06-07 15:54:46] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-ubuntu-18.04-amd64] Starting task.
[2024-06-07 15:55:30] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-ubuntu-18.04-amd64] Finished task in 44.19 seconds.
[2024-06-07 15:55:30] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-ubuntu-20.04-amd64] Starting task.
[2024-06-07 15:56:18] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-ubuntu-20.04-amd64] Finished task in 48.32 seconds.
[2024-06-07 15:56:19] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-debian-10-amd64] Starting task.
[2024-06-07 15:57:18] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-debian-10-amd64] Finished task in 59.94 seconds.
[2024-06-07 15:57:18] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-debian-11-amd64] Starting task.
[2024-06-07 15:58:02] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-debian-11-amd64] Finished task in 43.30 seconds.
[2024-06-07 15:58:02] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-debian-12-amd64] Starting task.
[2024-06-07 15:58:45] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-debian-12-amd64] Finished task in 43.49 seconds.
[2024-06-07 15:58:45] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-oracle-9-amd64] Starting task.
[2024-06-07 15:59:41] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-oracle-9-amd64] Finished task in 55.83 seconds.
[2024-06-07 15:59:41] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-ubuntu-18.04-amd64-tests] Starting task.
[2024-06-07 15:59:51] [ERROR] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-ubuntu-18.04-amd64-tests] Task failed with error: Error executing process task Traceback (most recent call last):
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/main.py", line 30, in <module>
    Tester.run(InputPayload(**vars(parse_arguments())))
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/testing.py", line 61, in run
    cls._run_tests(payload.tests, ansible, extra_vars)
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/testing.py", line 102, in _run_tests
    raise Exception(f"Test {test} failed with error: {event['stdout']}")
Exception: Test install failed with error: fatal: [localhost]: FAILED! => changed=true 
  cmd:
  - python3
  - -m
  - pytest
  - modules/testing/tests/test_agent/test_install.py
  - -v
  - --wazuh_version=4.7.4
  - --wazuh_revision=40717
  - --component=agent
  - --dependencies={}
  - '--targets={wazuh-1: /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml, agent: /tmp/dtt1-poc/agent-linux-ubuntu-18.04-amd64/inventory.yaml}'
  - --live=False
  - -s
  delta: '0:00:06.064940'
  end: '2024-06-07 15:59:51.150056'
  msg: non-zero return code
  rc: 1
  start: '2024-06-07 15:59:45.085116'
  stderr: |-
    Warning: Permanently added '192.168.57.111' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.111' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.111' (ED25519) to the list of known hosts.
    Unit wazuh-agent.service could not be found.
  stderr_lines: <omitted>
  stdout: |-
    [37m[2024-06-07 15:59:45] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'[0m
    [37m[2024-06-07 15:59:45] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'[0m
    ============================= test session starts ==============================
    platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /home/marcelo/.pyenv/versions/dtt1/bin/python3
    cachedir: .pytest_cache
    rootdir: /home/marcelo/wazuh/wazuh-qa/deployability/modules
    collecting ... collected 2 items

    modules/testing/tests/test_agent/test_install.py::test_installation [32m[2024-06-07 15:59:45] [INFO] TESTER: Checking connection to ubuntu-18.04[0m
    [32m[2024-06-07 15:59:45] [INFO] TESTER: Connection established successfully in ubuntu-18.04[0m
    [32m[2024-06-07 15:59:46] [INFO] TESTER: Checking connection to ubuntu-22.04[0m
    [32m[2024-06-07 15:59:46] [INFO] TESTER: Connection established successfully in ubuntu-22.04[0m
    [32m[2024-06-07 15:59:47] [INFO] TESTER: No Firewall to disable on ubuntu-18.04[0m
    [32m[2024-06-07 15:59:47] [INFO] TESTER: No Firewall to disable on ubuntu-22.04[0m
    FAILED
    modules/testing/tests/test_agent/test_install.py::test_status [32m[2024-06-07 15:59:50] [INFO] TESTER: Getting status of ubuntu-18.04[0m
    FAILED

    =================================== FAILURES ===================================
    ______________________________ test_installation _______________________________

    wazuh_params = {'agents': {'agent-linux-ubuntu-1804-amd64': '/tmp/dtt1-poc/agent-linux-ubuntu-18.04-amd64/inventory.yaml'}, 'dashboar...entory.yaml', 'dependencies': '{}', 'indexers': ['/tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml'], ...}

        def test_installation(wazuh_params):
            # Checking connection
            for _, manager_params in wazuh_params['managers'].items():
                Utils.check_inventory_connection(manager_params)

            # Certs creation, firewall management and Manager installation
            for agent_name, agent_params in wazuh_params['agents'].items():
                HostConfiguration.disable_firewall(agent_params)

            if HostInformation.dir_exists(wazuh_params['master'], WAZUH_ROOT):
                logger.info(f'Manager is already installed in {HostInformation.get_os_name_and_version_from_inventory(wazuh_params["master"])}')
            else:
                HostConfiguration.disable_firewall(manager_params)
                HostConfiguration.certs_create(wazuh_params['wazuh_version'], wazuh_params['master'], wazuh_params['dashboard'], wazuh_params['indexers'], wazuh_params['workers'])
    >           WazuhManager.install_manager(wazuh_params['master'], 'wazuh-1', wazuh_params['wazuh_version'])
    E           TypeError: WazuhManager.install_manager() missing 1 required positional argument: 'live'

    modules/testing/tests/test_agent/test_install.py:80: TypeError
    _________________________________ test_status __________________________________

    wazuh_params = {'agents': {'agent-linux-ubuntu-1804-amd64': '/tmp/dtt1-poc/agent-linux-ubuntu-18.04-amd64/inventory.yaml'}, 'dashboar...entory.yaml', 'dependencies': '{}', 'indexers': ['/tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml'], ...}

        def test_status(wazuh_params):
            for agent in wazuh_params['agents'].values():
                agent_status = GeneralComponentActions.get_component_status(agent, 'wazuh-agent')
                valid_statuses = ['loaded', 'Stopped', 'not running']
    >           assert any(valid_status in agent_status for valid_status in valid_statuses), logger.error(f'The {HostInformation.get_os_name_and_version_from_inventory(agent)} status is not loaded')

    modules/testing/tests/test_agent/test_install.py:104:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    .0 = <list_iterator object at 0x784d0feeac50>

    >   assert any(valid_status in agent_status for valid_status in valid_statuses), logger.error(f'The {HostInformation.get_os_name_and_version_from_inventory(agent)} status is not loaded')
    E   TypeError: argument of type 'NoneType' is not iterable

    modules/testing/tests/test_agent/test_install.py:104: TypeError
    =============================== warnings summary ===============================
    modules/provision/models.py:36
      /home/marcelo/wazuh/wazuh-qa/deployability/modules/provision/models.py:36: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
        @validator('dependencies', pre=True)

    modules/provision/models.py:64
      /home/marcelo/wazuh/wazuh-qa/deployability/modules/provision/models.py:64: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
        @validator('install', 'uninstall', pre=True)

    -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
    =========================== short test summary info ============================
    FAILED modules/testing/tests/test_agent/test_install.py::test_installation - ...
    FAILED modules/testing/tests/test_agent/test_install.py::test_status - TypeEr...
    ======================== 2 failed, 2 warnings in 5.54s =========================
  stdout_lines: <omitted>
.
[2024-06-07 15:59:51] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-ubuntu-20.04-amd64-tests] Starting task.
[2024-06-07 16:00:00] [ERROR] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-ubuntu-20.04-amd64-tests] Task failed with error: Error executing process task Traceback (most recent call last):
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/main.py", line 30, in <module>
    Tester.run(InputPayload(**vars(parse_arguments())))
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/testing.py", line 61, in run
    cls._run_tests(payload.tests, ansible, extra_vars)
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/testing.py", line 102, in _run_tests
    raise Exception(f"Test {test} failed with error: {event['stdout']}")
Exception: Test install failed with error: fatal: [localhost]: FAILED! => changed=true 
  cmd:
  - python3
  - -m
  - pytest
  - modules/testing/tests/test_agent/test_install.py
  - -v
  - --wazuh_version=4.7.4
  - --wazuh_revision=40717
  - --component=agent
  - --dependencies={}
  - '--targets={wazuh-1: /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml, agent: /tmp/dtt1-poc/agent-linux-ubuntu-20.04-amd64/inventory.yaml}'
  - --live=False
  - -s
  delta: '0:00:05.727667'
  end: '2024-06-07 16:00:00.389436'
  msg: non-zero return code
  rc: 1
  start: '2024-06-07 15:59:54.661769'
  stderr: |-
    Warning: Permanently added '192.168.57.160' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.160' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.160' (ED25519) to the list of known hosts.
    Unit wazuh-agent.service could not be found.
  stderr_lines: <omitted>
  stdout: |-
    [37m[2024-06-07 15:59:54] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'[0m
    [37m[2024-06-07 15:59:54] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'[0m
    ============================= test session starts ==============================
    platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /home/marcelo/.pyenv/versions/dtt1/bin/python3
    cachedir: .pytest_cache
    rootdir: /home/marcelo/wazuh/wazuh-qa/deployability/modules
    collecting ... collected 2 items

    modules/testing/tests/test_agent/test_install.py::test_installation [32m[2024-06-07 15:59:55] [INFO] TESTER: Checking connection to ubuntu-20.04[0m
    [32m[2024-06-07 15:59:55] [INFO] TESTER: Connection established successfully in ubuntu-20.04[0m
    [32m[2024-06-07 15:59:55] [INFO] TESTER: Checking connection to ubuntu-22.04[0m
    [32m[2024-06-07 15:59:56] [INFO] TESTER: Connection established successfully in ubuntu-22.04[0m
    [32m[2024-06-07 15:59:56] [INFO] TESTER: No Firewall to disable on ubuntu-20.04[0m
    [32m[2024-06-07 15:59:56] [INFO] TESTER: No Firewall to disable on ubuntu-22.04[0m
    FAILED
    modules/testing/tests/test_agent/test_install.py::test_status [32m[2024-06-07 15:59:59] [INFO] TESTER: Getting status of ubuntu-20.04[0m
    FAILED

    =================================== FAILURES ===================================
    ______________________________ test_installation _______________________________

    wazuh_params = {'agents': {'agent-linux-ubuntu-2004-amd64': '/tmp/dtt1-poc/agent-linux-ubuntu-20.04-amd64/inventory.yaml'}, 'dashboar...entory.yaml', 'dependencies': '{}', 'indexers': ['/tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml'], ...}

        def test_installation(wazuh_params):
            # Checking connection
            for _, manager_params in wazuh_params['managers'].items():
                Utils.check_inventory_connection(manager_params)

            # Certs creation, firewall management and Manager installation
            for agent_name, agent_params in wazuh_params['agents'].items():
                HostConfiguration.disable_firewall(agent_params)

            if HostInformation.dir_exists(wazuh_params['master'], WAZUH_ROOT):
                logger.info(f'Manager is already installed in {HostInformation.get_os_name_and_version_from_inventory(wazuh_params["master"])}')
            else:
                HostConfiguration.disable_firewall(manager_params)
                HostConfiguration.certs_create(wazuh_params['wazuh_version'], wazuh_params['master'], wazuh_params['dashboard'], wazuh_params['indexers'], wazuh_params['workers'])
    >           WazuhManager.install_manager(wazuh_params['master'], 'wazuh-1', wazuh_params['wazuh_version'])
    E           TypeError: WazuhManager.install_manager() missing 1 required positional argument: 'live'

    modules/testing/tests/test_agent/test_install.py:80: TypeError
    _________________________________ test_status __________________________________

    wazuh_params = {'agents': {'agent-linux-ubuntu-2004-amd64': '/tmp/dtt1-poc/agent-linux-ubuntu-20.04-amd64/inventory.yaml'}, 'dashboar...entory.yaml', 'dependencies': '{}', 'indexers': ['/tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml'], ...}

        def test_status(wazuh_params):
            for agent in wazuh_params['agents'].values():
                agent_status = GeneralComponentActions.get_component_status(agent, 'wazuh-agent')
                valid_statuses = ['loaded', 'Stopped', 'not running']
    >           assert any(valid_status in agent_status for valid_status in valid_statuses), logger.error(f'The {HostInformation.get_os_name_and_version_from_inventory(agent)} status is not loaded')

    modules/testing/tests/test_agent/test_install.py:104:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    .0 = <list_iterator object at 0x707905fa6d10>

    >   assert any(valid_status in agent_status for valid_status in valid_statuses), logger.error(f'The {HostInformation.get_os_name_and_version_from_inventory(agent)} status is not loaded')
    E   TypeError: argument of type 'NoneType' is not iterable

    modules/testing/tests/test_agent/test_install.py:104: TypeError
    =============================== warnings summary ===============================
    modules/provision/models.py:36
      /home/marcelo/wazuh/wazuh-qa/deployability/modules/provision/models.py:36: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
        @validator('dependencies', pre=True)

    modules/provision/models.py:64
      /home/marcelo/wazuh/wazuh-qa/deployability/modules/provision/models.py:64: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
        @validator('install', 'uninstall', pre=True)

    -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
    =========================== short test summary info ============================
    FAILED modules/testing/tests/test_agent/test_install.py::test_installation - ...
    FAILED modules/testing/tests/test_agent/test_install.py::test_status - TypeEr...
    ======================== 2 failed, 2 warnings in 5.23s =========================
  stdout_lines: <omitted>
.
[2024-06-07 16:00:00] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-debian-10-amd64-tests] Starting task.
[2024-06-07 16:00:09] [ERROR] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-debian-10-amd64-tests] Task failed with error: Error executing process task Traceback (most recent call last):
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/main.py", line 30, in <module>
    Tester.run(InputPayload(**vars(parse_arguments())))
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/testing.py", line 61, in run
    cls._run_tests(payload.tests, ansible, extra_vars)
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/testing.py", line 102, in _run_tests
    raise Exception(f"Test {test} failed with error: {event['stdout']}")
Exception: Test install failed with error: fatal: [localhost]: FAILED! => changed=true 
  cmd:
  - python3
  - -m
  - pytest
  - modules/testing/tests/test_agent/test_install.py
  - -v
  - --wazuh_version=4.7.4
  - --wazuh_revision=40717
  - --component=agent
  - --dependencies={}
  - '--targets={wazuh-1: /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml, agent: /tmp/dtt1-poc/agent-linux-debian-10-amd64/inventory.yaml}'
  - --live=False
  - -s
  delta: '0:00:05.080992'
  end: '2024-06-07 16:00:08.885341'
  msg: non-zero return code
  rc: 1
  start: '2024-06-07 16:00:03.804349'
  stderr: |-
    Warning: Permanently added '192.168.57.186' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.186' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.186' (ED25519) to the list of known hosts.
    Unit wazuh-agent.service could not be found.
  stderr_lines: <omitted>
  stdout: |-
    [37m[2024-06-07 16:00:04] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'[0m
    [37m[2024-06-07 16:00:04] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'[0m
    ============================= test session starts ==============================
    platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /home/marcelo/.pyenv/versions/dtt1/bin/python3
    cachedir: .pytest_cache
    rootdir: /home/marcelo/wazuh/wazuh-qa/deployability/modules
    collecting ... collected 2 items

    modules/testing/tests/test_agent/test_install.py::test_installation [32m[2024-06-07 16:00:04] [INFO] TESTER: Checking connection to debian-10[0m
    [32m[2024-06-07 16:00:04] [INFO] TESTER: Connection established successfully in debian-10[0m
    [32m[2024-06-07 16:00:04] [INFO] TESTER: Checking connection to ubuntu-22.04[0m
    [32m[2024-06-07 16:00:04] [INFO] TESTER: Connection established successfully in ubuntu-22.04[0m
    [32m[2024-06-07 16:00:05] [INFO] TESTER: No Firewall to disable on debian-10[0m
    [32m[2024-06-07 16:00:05] [INFO] TESTER: No Firewall to disable on ubuntu-22.04[0m
    FAILED
    modules/testing/tests/test_agent/test_install.py::test_status [32m[2024-06-07 16:00:08] [INFO] TESTER: Getting status of debian-10[0m
    FAILED

    =================================== FAILURES ===================================
    ______________________________ test_installation _______________________________

    wazuh_params = {'agents': {'agent-linux-debian-10-amd64': '/tmp/dtt1-poc/agent-linux-debian-10-amd64/inventory.yaml'}, 'dashboard': '...entory.yaml', 'dependencies': '{}', 'indexers': ['/tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml'], ...}

        def test_installation(wazuh_params):
            # Checking connection
            for _, manager_params in wazuh_params['managers'].items():
                Utils.check_inventory_connection(manager_params)

            # Certs creation, firewall management and Manager installation
            for agent_name, agent_params in wazuh_params['agents'].items():
                HostConfiguration.disable_firewall(agent_params)

            if HostInformation.dir_exists(wazuh_params['master'], WAZUH_ROOT):
                logger.info(f'Manager is already installed in {HostInformation.get_os_name_and_version_from_inventory(wazuh_params["master"])}')
            else:
                HostConfiguration.disable_firewall(manager_params)
                HostConfiguration.certs_create(wazuh_params['wazuh_version'], wazuh_params['master'], wazuh_params['dashboard'], wazuh_params['indexers'], wazuh_params['workers'])
    >           WazuhManager.install_manager(wazuh_params['master'], 'wazuh-1', wazuh_params['wazuh_version'])
    E           TypeError: WazuhManager.install_manager() missing 1 required positional argument: 'live'

    modules/testing/tests/test_agent/test_install.py:80: TypeError
    _________________________________ test_status __________________________________

    wazuh_params = {'agents': {'agent-linux-debian-10-amd64': '/tmp/dtt1-poc/agent-linux-debian-10-amd64/inventory.yaml'}, 'dashboard': '...entory.yaml', 'dependencies': '{}', 'indexers': ['/tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml'], ...}

        def test_status(wazuh_params):
            for agent in wazuh_params['agents'].values():
                agent_status = GeneralComponentActions.get_component_status(agent, 'wazuh-agent')
                valid_statuses = ['loaded', 'Stopped', 'not running']
    >           assert any(valid_status in agent_status for valid_status in valid_statuses), logger.error(f'The {HostInformation.get_os_name_and_version_from_inventory(agent)} status is not loaded')

    modules/testing/tests/test_agent/test_install.py:104:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    .0 = <list_iterator object at 0x7ec7122e2d40>

    >   assert any(valid_status in agent_status for valid_status in valid_statuses), logger.error(f'The {HostInformation.get_os_name_and_version_from_inventory(agent)} status is not loaded')
    E   TypeError: argument of type 'NoneType' is not iterable

    modules/testing/tests/test_agent/test_install.py:104: TypeError
    =============================== warnings summary ===============================
    modules/provision/models.py:36
      /home/marcelo/wazuh/wazuh-qa/deployability/modules/provision/models.py:36: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
        @validator('dependencies', pre=True)

    modules/provision/models.py:64
      /home/marcelo/wazuh/wazuh-qa/deployability/modules/provision/models.py:64: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
        @validator('install', 'uninstall', pre=True)

    -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
    =========================== short test summary info ============================
    FAILED modules/testing/tests/test_agent/test_install.py::test_installation - ...
    FAILED modules/testing/tests/test_agent/test_install.py::test_status - TypeEr...
    ======================== 2 failed, 2 warnings in 4.60s =========================
  stdout_lines: <omitted>
.
[2024-06-07 16:00:09] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-debian-11-amd64-tests] Starting task.
[2024-06-07 16:00:18] [ERROR] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-debian-11-amd64-tests] Task failed with error: Error executing process task Traceback (most recent call last):
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/main.py", line 30, in <module>
    Tester.run(InputPayload(**vars(parse_arguments())))
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/testing.py", line 61, in run
    cls._run_tests(payload.tests, ansible, extra_vars)
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/testing.py", line 102, in _run_tests
    raise Exception(f"Test {test} failed with error: {event['stdout']}")
Exception: Test install failed with error: fatal: [localhost]: FAILED! => changed=true 
  cmd:
  - python3
  - -m
  - pytest
  - modules/testing/tests/test_agent/test_install.py
  - -v
  - --wazuh_version=4.7.4
  - --wazuh_revision=40717
  - --component=agent
  - --dependencies={}
  - '--targets={wazuh-1: /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml, agent: /tmp/dtt1-poc/agent-linux-debian-11-amd64/inventory.yaml}'
  - --live=False
  - -s
  delta: '0:00:05.921619'
  end: '2024-06-07 16:00:18.263097'
  msg: non-zero return code
  rc: 1
  start: '2024-06-07 16:00:12.341478'
  stderr: |-
    Warning: Permanently added '192.168.57.115' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.115' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.115' (ED25519) to the list of known hosts.
    Unit wazuh-agent.service could not be found.
  stderr_lines: <omitted>
  stdout: |-
    [37m[2024-06-07 16:00:12] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'[0m
    [37m[2024-06-07 16:00:12] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'[0m
    ============================= test session starts ==============================
    platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /home/marcelo/.pyenv/versions/dtt1/bin/python3
    cachedir: .pytest_cache
    rootdir: /home/marcelo/wazuh/wazuh-qa/deployability/modules
    collecting ... collected 2 items

    modules/testing/tests/test_agent/test_install.py::test_installation [32m[2024-06-07 16:00:12] [INFO] TESTER: Checking connection to debian-11[0m
    [32m[2024-06-07 16:00:13] [INFO] TESTER: Connection established successfully in debian-11[0m
    [32m[2024-06-07 16:00:13] [INFO] TESTER: Checking connection to ubuntu-22.04[0m
    [32m[2024-06-07 16:00:13] [INFO] TESTER: Connection established successfully in ubuntu-22.04[0m
    [32m[2024-06-07 16:00:13] [INFO] TESTER: No Firewall to disable on debian-11[0m
    [32m[2024-06-07 16:00:13] [INFO] TESTER: No Firewall to disable on ubuntu-22.04[0m
    FAILED
    modules/testing/tests/test_agent/test_install.py::test_status [32m[2024-06-07 16:00:18] [INFO] TESTER: Getting status of debian-11[0m
    FAILED

    =================================== FAILURES ===================================
    ______________________________ test_installation _______________________________

    wazuh_params = {'agents': {'agent-linux-debian-11-amd64': '/tmp/dtt1-poc/agent-linux-debian-11-amd64/inventory.yaml'}, 'dashboard': '...entory.yaml', 'dependencies': '{}', 'indexers': ['/tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml'], ...}

        def test_installation(wazuh_params):
            # Checking connection
            for _, manager_params in wazuh_params['managers'].items():
                Utils.check_inventory_connection(manager_params)

            # Certs creation, firewall management and Manager installation
            for agent_name, agent_params in wazuh_params['agents'].items():
                HostConfiguration.disable_firewall(agent_params)

            if HostInformation.dir_exists(wazuh_params['master'], WAZUH_ROOT):
                logger.info(f'Manager is already installed in {HostInformation.get_os_name_and_version_from_inventory(wazuh_params["master"])}')
            else:
                HostConfiguration.disable_firewall(manager_params)
                HostConfiguration.certs_create(wazuh_params['wazuh_version'], wazuh_params['master'], wazuh_params['dashboard'], wazuh_params['indexers'], wazuh_params['workers'])
    >           WazuhManager.install_manager(wazuh_params['master'], 'wazuh-1', wazuh_params['wazuh_version'])
    E           TypeError: WazuhManager.install_manager() missing 1 required positional argument: 'live'

    modules/testing/tests/test_agent/test_install.py:80: TypeError
    _________________________________ test_status __________________________________

    wazuh_params = {'agents': {'agent-linux-debian-11-amd64': '/tmp/dtt1-poc/agent-linux-debian-11-amd64/inventory.yaml'}, 'dashboard': '...entory.yaml', 'dependencies': '{}', 'indexers': ['/tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml'], ...}

        def test_status(wazuh_params):
            for agent in wazuh_params['agents'].values():
                agent_status = GeneralComponentActions.get_component_status(agent, 'wazuh-agent')
                valid_statuses = ['loaded', 'Stopped', 'not running']
    >           assert any(valid_status in agent_status for valid_status in valid_statuses), logger.error(f'The {HostInformation.get_os_name_and_version_from_inventory(agent)} status is not loaded')

    modules/testing/tests/test_agent/test_install.py:104:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    .0 = <list_iterator object at 0x7496dfaeac50>

    >   assert any(valid_status in agent_status for valid_status in valid_statuses), logger.error(f'The {HostInformation.get_os_name_and_version_from_inventory(agent)} status is not loaded')
    E   TypeError: argument of type 'NoneType' is not iterable

    modules/testing/tests/test_agent/test_install.py:104: TypeError
    =============================== warnings summary ===============================
    modules/provision/models.py:36
      /home/marcelo/wazuh/wazuh-qa/deployability/modules/provision/models.py:36: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
        @validator('dependencies', pre=True)

    modules/provision/models.py:64
      /home/marcelo/wazuh/wazuh-qa/deployability/modules/provision/models.py:64: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
        @validator('install', 'uninstall', pre=True)

    -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
    =========================== short test summary info ============================
    FAILED modules/testing/tests/test_agent/test_install.py::test_installation - ...
    FAILED modules/testing/tests/test_agent/test_install.py::test_status - TypeEr...
    ======================== 2 failed, 2 warnings in 5.43s =========================
  stdout_lines: <omitted>
.
[2024-06-07 16:00:18] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-debian-12-amd64-tests] Starting task.
[2024-06-07 16:00:27] [ERROR] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-debian-12-amd64-tests] Task failed with error: Error executing process task Traceback (most recent call last):
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/main.py", line 30, in <module>
    Tester.run(InputPayload(**vars(parse_arguments())))
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/testing.py", line 61, in run
    cls._run_tests(payload.tests, ansible, extra_vars)
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/testing.py", line 102, in _run_tests
    raise Exception(f"Test {test} failed with error: {event['stdout']}")
Exception: Test install failed with error: fatal: [localhost]: FAILED! => changed=true 
  cmd:
  - python3
  - -m
  - pytest
  - modules/testing/tests/test_agent/test_install.py
  - -v
  - --wazuh_version=4.7.4
  - --wazuh_revision=40717
  - --component=agent
  - --dependencies={}
  - '--targets={wazuh-1: /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml, agent: /tmp/dtt1-poc/agent-linux-debian-12-amd64/inventory.yaml}'
  - --live=False
  - -s
  delta: '0:00:05.152465'
  end: '2024-06-07 16:00:26.906472'
  msg: non-zero return code
  rc: 1
  start: '2024-06-07 16:00:21.754007'
  stderr: |-
    Warning: Permanently added '192.168.57.121' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.121' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.121' (ED25519) to the list of known hosts.
    Unit wazuh-agent.service could not be found.
  stderr_lines: <omitted>
  stdout: |-
    [37m[2024-06-07 16:00:22] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'[0m
    [37m[2024-06-07 16:00:22] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'[0m
    ============================= test session starts ==============================
    platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /home/marcelo/.pyenv/versions/dtt1/bin/python3
    cachedir: .pytest_cache
    rootdir: /home/marcelo/wazuh/wazuh-qa/deployability/modules
    collecting ... collected 2 items

    modules/testing/tests/test_agent/test_install.py::test_installation [32m[2024-06-07 16:00:22] [INFO] TESTER: Checking connection to debian-12[0m
    [32m[2024-06-07 16:00:22] [INFO] TESTER: Connection established successfully in debian-12[0m
    [32m[2024-06-07 16:00:22] [INFO] TESTER: Checking connection to ubuntu-22.04[0m
    [32m[2024-06-07 16:00:22] [INFO] TESTER: Connection established successfully in ubuntu-22.04[0m
    [32m[2024-06-07 16:00:22] [INFO] TESTER: No Firewall to disable on debian-12[0m
    [32m[2024-06-07 16:00:23] [INFO] TESTER: No Firewall to disable on ubuntu-22.04[0m
    FAILED
    modules/testing/tests/test_agent/test_install.py::test_status [32m[2024-06-07 16:00:26] [INFO] TESTER: Getting status of debian-12[0m
    FAILED

    =================================== FAILURES ===================================
    ______________________________ test_installation _______________________________

    wazuh_params = {'agents': {'agent-linux-debian-12-amd64': '/tmp/dtt1-poc/agent-linux-debian-12-amd64/inventory.yaml'}, 'dashboard': '...entory.yaml', 'dependencies': '{}', 'indexers': ['/tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml'], ...}

        def test_installation(wazuh_params):
            # Checking connection
            for _, manager_params in wazuh_params['managers'].items():
                Utils.check_inventory_connection(manager_params)

            # Certs creation, firewall management and Manager installation
            for agent_name, agent_params in wazuh_params['agents'].items():
                HostConfiguration.disable_firewall(agent_params)

            if HostInformation.dir_exists(wazuh_params['master'], WAZUH_ROOT):
                logger.info(f'Manager is already installed in {HostInformation.get_os_name_and_version_from_inventory(wazuh_params["master"])}')
            else:
                HostConfiguration.disable_firewall(manager_params)
                HostConfiguration.certs_create(wazuh_params['wazuh_version'], wazuh_params['master'], wazuh_params['dashboard'], wazuh_params['indexers'], wazuh_params['workers'])
    >           WazuhManager.install_manager(wazuh_params['master'], 'wazuh-1', wazuh_params['wazuh_version'])
    E           TypeError: WazuhManager.install_manager() missing 1 required positional argument: 'live'

    modules/testing/tests/test_agent/test_install.py:80: TypeError
    _________________________________ test_status __________________________________

    wazuh_params = {'agents': {'agent-linux-debian-12-amd64': '/tmp/dtt1-poc/agent-linux-debian-12-amd64/inventory.yaml'}, 'dashboard': '...entory.yaml', 'dependencies': '{}', 'indexers': ['/tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml'], ...}

        def test_status(wazuh_params):
            for agent in wazuh_params['agents'].values():
                agent_status = GeneralComponentActions.get_component_status(agent, 'wazuh-agent')
                valid_statuses = ['loaded', 'Stopped', 'not running']
    >           assert any(valid_status in agent_status for valid_status in valid_statuses), logger.error(f'The {HostInformation.get_os_name_and_version_from_inventory(agent)} status is not loaded')

    modules/testing/tests/test_agent/test_install.py:104:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    .0 = <list_iterator object at 0x71292efa2d70>

    >   assert any(valid_status in agent_status for valid_status in valid_statuses), logger.error(f'The {HostInformation.get_os_name_and_version_from_inventory(agent)} status is not loaded')
    E   TypeError: argument of type 'NoneType' is not iterable

    modules/testing/tests/test_agent/test_install.py:104: TypeError
    =============================== warnings summary ===============================
    modules/provision/models.py:36
      /home/marcelo/wazuh/wazuh-qa/deployability/modules/provision/models.py:36: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
        @validator('dependencies', pre=True)

    modules/provision/models.py:64
      /home/marcelo/wazuh/wazuh-qa/deployability/modules/provision/models.py:64: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
        @validator('install', 'uninstall', pre=True)

    -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
    =========================== short test summary info ============================
    FAILED modules/testing/tests/test_agent/test_install.py::test_installation - ...
    FAILED modules/testing/tests/test_agent/test_install.py::test_status - TypeEr...
    ======================== 2 failed, 2 warnings in 4.66s =========================
  stdout_lines: <omitted>
.
[2024-06-07 16:00:27] [INFO] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-oracle-9-amd64-tests] Starting task.
[2024-06-07 16:00:37] [ERROR] [10222] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-oracle-9-amd64-tests] Task failed with error: Error executing process task Traceback (most recent call last):
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/main.py", line 30, in <module>
    Tester.run(InputPayload(**vars(parse_arguments())))
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/testing.py", line 61, in run
    cls._run_tests(payload.tests, ansible, extra_vars)
  File "/home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/testing.py", line 102, in _run_tests
    raise Exception(f"Test {test} failed with error: {event['stdout']}")
Exception: Test install failed with error: fatal: [localhost]: FAILED! => changed=true 
  cmd:
  - python3
  - -m
  - pytest
  - modules/testing/tests/test_agent/test_install.py
  - -v
  - --wazuh_version=4.7.4
  - --wazuh_revision=40717
  - --component=agent
  - --dependencies={}
  - '--targets={wazuh-1: /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml, agent: /tmp/dtt1-poc/agent-linux-oracle-9-amd64/inventory.yaml}'
  - --live=False
  - -s
  delta: '0:00:06.572164'
  end: '2024-06-07 16:00:36.977804'
  msg: non-zero return code
  rc: 1
  start: '2024-06-07 16:00:30.405640'
  stderr: |-
    Warning: Permanently added '192.168.57.60' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.60' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.60' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.60' (ED25519) to the list of known hosts.
    Removed "/etc/systemd/system/multi-user.target.wants/firewalld.service".
    Removed "/etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service".
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.156' (ED25519) to the list of known hosts.
    Warning: Permanently added '192.168.57.60' (ED25519) to the list of known hosts.
    Unit wazuh-agent.service could not be found.
  stderr_lines: <omitted>
  stdout: |-
    [37m[2024-06-07 16:00:30] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'[0m
    [37m[2024-06-07 16:00:30] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'[0m
    ============================= test session starts ==============================
    platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /home/marcelo/.pyenv/versions/dtt1/bin/python3
    cachedir: .pytest_cache
    rootdir: /home/marcelo/wazuh/wazuh-qa/deployability/modules
    collecting ... collected 2 items

    modules/testing/tests/test_agent/test_install.py::test_installation [32m[2024-06-07 16:00:30] [INFO] TESTER: Checking connection to oracle-9[0m
    [32m[2024-06-07 16:00:31] [INFO] TESTER: Connection established successfully in oracle-9[0m
    [32m[2024-06-07 16:00:31] [INFO] TESTER: Checking connection to ubuntu-22.04[0m
    [32m[2024-06-07 16:00:31] [INFO] TESTER: Connection established successfully in ubuntu-22.04[0m
    [32m[2024-06-07 16:00:32] [INFO] TESTER: Firewall disabled on oracle-9[0m
    [32m[2024-06-07 16:00:33] [INFO] TESTER: No Firewall to disable on ubuntu-22.04[0m
    FAILED
    modules/testing/tests/test_agent/test_install.py::test_status [32m[2024-06-07 16:00:36] [INFO] TESTER: Getting status of oracle-9[0m
    FAILED

    =================================== FAILURES ===================================
    ______________________________ test_installation _______________________________

    wazuh_params = {'agents': {'agent-linux-oracle-9-amd64': '/tmp/dtt1-poc/agent-linux-oracle-9-amd64/inventory.yaml'}, 'dashboard': '/t...entory.yaml', 'dependencies': '{}', 'indexers': ['/tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml'], ...}

        def test_installation(wazuh_params):
            # Checking connection
            for _, manager_params in wazuh_params['managers'].items():
                Utils.check_inventory_connection(manager_params)

            # Certs creation, firewall management and Manager installation
            for agent_name, agent_params in wazuh_params['agents'].items():
                HostConfiguration.disable_firewall(agent_params)

            if HostInformation.dir_exists(wazuh_params['master'], WAZUH_ROOT):
                logger.info(f'Manager is already installed in {HostInformation.get_os_name_and_version_from_inventory(wazuh_params["master"])}')
            else:
                HostConfiguration.disable_firewall(manager_params)
                HostConfiguration.certs_create(wazuh_params['wazuh_version'], wazuh_params['master'], wazuh_params['dashboard'], wazuh_params['indexers'], wazuh_params['workers'])
    >           WazuhManager.install_manager(wazuh_params['master'], 'wazuh-1', wazuh_params['wazuh_version'])
    E           TypeError: WazuhManager.install_manager() missing 1 required positional argument: 'live'

    modules/testing/tests/test_agent/test_install.py:80: TypeError
    _________________________________ test_status __________________________________

    wazuh_params = {'agents': {'agent-linux-oracle-9-amd64': '/tmp/dtt1-poc/agent-linux-oracle-9-amd64/inventory.yaml'}, 'dashboard': '/t...entory.yaml', 'dependencies': '{}', 'indexers': ['/tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml'], ...}

        def test_status(wazuh_params):
            for agent in wazuh_params['agents'].values():
                agent_status = GeneralComponentActions.get_component_status(agent, 'wazuh-agent')
                valid_statuses = ['loaded', 'Stopped', 'not running']
    >           assert any(valid_status in agent_status for valid_status in valid_statuses), logger.error(f'The {HostInformation.get_os_name_and_version_from_inventory(agent)} status is not loaded')

    modules/testing/tests/test_agent/test_install.py:104:
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    .0 = <list_iterator object at 0x777f496fecb0>

    >   assert any(valid_status in agent_status for valid_status in valid_statuses), logger.error(f'The {HostInformation.get_os_name_and_version_from_inventory(agent)} status is not loaded')
    E   TypeError: argument of type 'NoneType' is not iterable

    modules/testing/tests/test_agent/test_install.py:104: TypeError
    =============================== warnings summary ===============================
    modules/provision/models.py:36
      /home/marcelo/wazuh/wazuh-qa/deployability/modules/provision/models.py:36: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
        @validator('dependencies', pre=True)

    modules/provision/models.py:64
      /home/marcelo/wazuh/wazuh-qa/deployability/modules/provision/models.py:64: PydanticDeprecatedSince20: Pydantic V1 style `@validator` validators are deprecated. You should migrate to Pydantic V2 style `@field_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.5/migration/
        @validator('install', 'uninstall', pre=True)

    -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
    =========================== short test summary info ============================
    FAILED modules/testing/tests/test_agent/test_install.py::test_installation - ...
    FAILED modules/testing/tests/test_agent/test_install.py::test_status - TypeEr...
    ======================== 2 failed, 2 warnings in 6.07s =========================
  stdout_lines: <omitted>
.
[2024-06-07 16:00:37] [INFO] [10222] [MainThread] [jobflow]: Executing Reverse DAG tasks.
[2024-06-07 16:00:37] [INFO] [10222] [MainThread] [jobflow]: Executing tasks in parallel.
[2024-06-07 16:00:37] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-ubuntu-18.04-amd64] Starting task.
[2024-06-07 16:00:41] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-ubuntu-18.04-amd64] Finished task in 4.73 seconds.
[2024-06-07 16:00:41] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-ubuntu-20.04-amd64] Starting task.
[2024-06-07 16:00:46] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-ubuntu-20.04-amd64] Finished task in 4.70 seconds.
[2024-06-07 16:00:46] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-debian-10-amd64] Starting task.
[2024-06-07 16:00:51] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-debian-10-amd64] Finished task in 4.89 seconds.
[2024-06-07 16:00:51] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-debian-11-amd64] Starting task.
[2024-06-07 16:00:55] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-debian-11-amd64] Finished task in 4.39 seconds.
[2024-06-07 16:00:55] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-debian-12-amd64] Starting task.
[2024-06-07 16:01:00] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-debian-12-amd64] Finished task in 4.36 seconds.
[2024-06-07 16:01:00] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-oracle-9-amd64] Starting task.
[2024-06-07 16:01:05] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-oracle-9-amd64] Finished task in 5.02 seconds.
[2024-06-07 16:01:05] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-manager-linux-ubuntu-22.04-amd64] Starting task.
[2024-06-07 16:01:10] [INFO] [10222] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-manager-linux-ubuntu-22.04-amd64] Finished task in 4.76 seconds.

The failures are not related to jobflow, are related to other dtt modules.

mhamra commented 4 months ago

UPDATE

I've renamed all workflow references to jobflow, including class names, variables, documentation, docstrings, etc.

This is the result obtained after reinstalling the package and running a jobflow file:

(dtt1) marcelo@marcelo-B460-AORUS-PRO-AC:~/wazuh/wazuh-qa/deployability$ python -m jobflow modules/jobflow/examples/agent/vagrant/test-agent-complete-1.yaml 
[2024-06-10 15:18:54] [INFO] [121581] [MainThread] [jobflow]: Executing DAG tasks.
[2024-06-10 15:18:54] [INFO] [121581] [MainThread] [jobflow]: Executing tasks in parallel.
[2024-06-10 15:18:54] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-manager-linux-ubuntu-22.04-amd64] Starting task.
[2024-06-10 15:19:39] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-manager-linux-ubuntu-22.04-amd64] Finished task in 45.21 seconds.
[2024-06-10 15:19:39] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [provision-manager-linux-ubuntu-22.04-amd64] Starting task.
[2024-06-10 15:30:37] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [provision-manager-linux-ubuntu-22.04-amd64] Finished task in 658.46 seconds.
[2024-06-10 15:30:37] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-ubuntu-18.04-amd64] Starting task.
[2024-06-10 15:31:23] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-ubuntu-18.04-amd64] Finished task in 45.52 seconds.
[2024-06-10 15:31:23] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-ubuntu-20.04-amd64] Starting task.
[2024-06-10 15:32:10] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-ubuntu-20.04-amd64] Finished task in 47.25 seconds.
[2024-06-10 15:32:10] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-debian-10-amd64] Starting task.
[2024-06-10 15:33:18] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-debian-10-amd64] Finished task in 67.99 seconds.
[2024-06-10 15:33:18] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-debian-11-amd64] Starting task.
[2024-06-10 15:34:02] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-debian-11-amd64] Finished task in 44.02 seconds.
[2024-06-10 15:34:02] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-debian-12-amd64] Starting task.
[2024-06-10 15:34:47] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-debian-12-amd64] Finished task in 44.95 seconds.
[2024-06-10 15:34:47] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-oracle-9-amd64] Starting task.
[2024-06-10 15:35:42] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [allocate-agent-linux-oracle-9-amd64] Finished task in 55.25 seconds.
[2024-06-10 15:35:42] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-ubuntu-18.04-amd64-tests] Starting task.
[2024-06-10 15:37:49] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-ubuntu-18.04-amd64-tests] Finished task in 126.81 seconds.
[2024-06-10 15:37:49] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-ubuntu-20.04-amd64-tests] Starting task.
[2024-06-10 15:39:56] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-ubuntu-20.04-amd64-tests] Finished task in 126.72 seconds.
[2024-06-10 15:39:56] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-debian-10-amd64-tests] Starting task.
[2024-06-10 15:41:49] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-debian-10-amd64-tests] Finished task in 112.58 seconds.
[2024-06-10 15:41:49] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-debian-11-amd64-tests] Starting task.
[2024-06-10 15:43:36] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-debian-11-amd64-tests] Finished task in 107.78 seconds.
[2024-06-10 15:43:36] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-oracle-9-amd64-tests] Starting task.
[2024-06-10 15:46:22] [INFO] [121581] [ThreadPoolExecutor-0_0] [jobflow]: [run-agent-linux-oracle-9-amd64-tests] Finished task in 166.03 seconds.
[2024-06-10 15:46:22] [INFO] [121581] [MainThread] [jobflow]: Executing Reverse DAG tasks.
[2024-06-10 15:46:22] [INFO] [121581] [MainThread] [jobflow]: Executing tasks in parallel.
[2024-06-10 15:46:22] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-ubuntu-18.04-amd64] Starting task.
[2024-06-10 15:46:27] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-ubuntu-18.04-amd64] Finished task in 4.88 seconds.
[2024-06-10 15:46:27] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-ubuntu-20.04-amd64] Starting task.
[2024-06-10 15:46:32] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-ubuntu-20.04-amd64] Finished task in 4.91 seconds.
[2024-06-10 15:46:32] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-debian-10-amd64] Starting task.
[2024-06-10 15:46:37] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-debian-10-amd64] Finished task in 4.79 seconds.
[2024-06-10 15:46:37] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-debian-11-amd64] Starting task.
[2024-06-10 15:46:42] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-debian-11-amd64] Finished task in 4.44 seconds.
[2024-06-10 15:46:42] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-debian-12-amd64] Starting task.
[2024-06-10 15:46:46] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-debian-12-amd64] Finished task in 4.41 seconds.
[2024-06-10 15:46:46] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-oracle-9-amd64] Starting task.
[2024-06-10 15:46:51] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-agent-linux-oracle-9-amd64] Finished task in 5.11 seconds.
[2024-06-10 15:46:51] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-manager-linux-ubuntu-22.04-amd64] Starting task.
[2024-06-10 15:46:56] [INFO] [121581] [ThreadPoolExecutor-1_0] [jobflow]: [allocate-manager-linux-ubuntu-22.04-amd64] Finished task in 4.97 seconds.

UNIT TESTS

(dtt1) marcelo@marcelo-B460-AORUS-PRO-AC:~/wazuh/wazuh-qa/deployability$ pytest modules/jobflow
================================================================================= test session starts ==================================================================================
platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0
rootdir: /home/marcelo/wazuh/wazuh-qa/deployability/modules
collected 89 items                                                                                                                                                                     

modules/jobflow/tests/test_dag.py ..................................                                                                                                             [ 38%]
modules/jobflow/tests/test_jobflow_file.py ................                                                                                                                      [ 56%]
modules/jobflow/tests/test_jobflow_processor.py ......................                                                                                                           [ 80%]
modules/jobflow/tests/test_schema_validator.py .......                                                                                                                           [ 88%]
modules/jobflow/tests/test_task.py ..........                                                                                                                                    [100%]

================================================================================== 89 passed in 0.35s ==================================================================================

mhamra commented 4 months ago