c-bordon
commented
2 weeks ago This error only affects local deployments with Vagrant, since IP availability validation is done for this type of deployment. I understand that this was originally done to manage which IPs are assigned to the local machine and then configure it in the inventory.yml file to be able to connect to the VM.
The problem occurs because currently, the Allocator module handles the deployment of individual machines, as it runs in 3 threads, each execution validates the availability of the first IP 192.168.57.2, and when it responds that the IP is free, it proceeds to try to configure it.
The fix that occurs to me for this is to directly remove this method and configure the Vagrant VM network in DHCP, so that Vagrant and VirtualBox are in charge of configuring the IP and removing this responsibility from the Allocator. Then we can get the IP of the VM to configure it in the inventory.yml file
fcaffieri
Description
Running this workflow file with the
--threads 3workflow.yaml
``` version: 0.1 description: This workflow is used to test manager deployment for DDT1 PoC variables: manager-os: - linux-ubuntu-20.04-amd64 - linux-ubuntu-22.04-amd64 - linux-oracle-9-amd64 # - linux-amazon-2-amd64 # - linux-redhat-7-amd64 # - linux-redhat-8-amd64 # - linux-redhat-9-amd64 # - linux-centos-7-amd64 # - linux-centos-8-amd64 # - linux-debian-10-amd64 # - linux-debian-11-amd64 # - linux-debian-12-amd64 infra-provider: vagrant working-dir: /tmp/dtt1-poc tasks: # Unique manager allocate task - task: "allocate-manager-{manager}" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "{manager}" - inventory-output: "{working-dir}/manager-{manager}/inventory.yaml" - track-output: "{working-dir}/manager-{manager}/track.yaml" on-error: "abort-all" foreach: - variable: manager-os as: manager # Generic manager test task - task: "run-manager-tests" description: "Run tests install for the manager." do: this: process with: path: python3 args: - modules/testing/main.py - targets: - wazuh-1: "{working-dir}/manager-linux-ubuntu-20.04-amd64/inventory.yaml" - wazuh-2: "{working-dir}/manager-linux-ubuntu-22.04-amd64/inventory.yaml" - wazuh-3: "{working-dir}/manager-linux-oracle-9-amd64/inventory.yaml" # - wazuh-4: "{working-dir}/manager-linux-centos-7-amd64/inventory.yaml" # - wazuh-5: "{working-dir}/manager-linux-amazon-2-amd64/inventory.yaml" # - wazuh-6: "{working-dir}/manager-linux-redhat-7-amd64/inventory.yaml" # - wazuh-7: "{working-dir}/manager-linux-redhat-8-amd64/inventory.yaml" # - wazuh-8: "{working-dir}/manager-linux-redhat-9-amd64/inventory.yaml" # - wazuh-9: "{working-dir}/manager-linux-centos-8-amd64/inventory.yaml" # - wazuh-10: "{working-dir}/mmarcelo@marcelo-B460-AORUS-PRO-AC:~/wazuh/wazuh-qa$ tail -f /tmp/workflow.log [2024-04-17 12:15:59,960] [INFO] [744222] [MainThread] [workflow_engine]: Executing DAG tasks. [2024-04-17 12:15:59,960] [INFO] [744222] [MainThread] [workflow_engine]: Executing tasks in parallel. [2024-04-17 12:15:59,960] [INFO] [744222] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-manager-linux-ubuntu-20.04-amd64] Starting task. [2024-04-17 12:15:59,961] [INFO] [744222] [ThreadPoolExecutor-0_1] [workflow_engine]: [allocate-manager-linux-ubuntu-22.04-amd64] Starting task. [2024-04-17 12:15:59,961] [INFO] [744222] [ThreadPoolExecutor-0_2] [workflow_engine]: [allocate-manager-linux-oracle-9-amd64] Starting task. [2024-04-17 12:16:00,188] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa [2024-04-17 12:16:00,189] [DEBUG] ALLOCATOR: No config provided. Generating from payload [2024-04-17 12:16:00,189] [DEBUG] ALLOCATOR: Generating new key pair [2024-04-17 12:16:00,192] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa [2024-04-17 12:16:00,193] [DEBUG] ALLOCATOR: No config provided. Generating from payload [2024-04-17 12:16:00,193] [DEBUG] ALLOCATOR: Generating new key pair [2024-04-17 12:16:00,214] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa [2024-04-17 12:16:00,214] [DEBUG] ALLOCATOR: No config provided. Generating from payload [2024-04-17 12:16:00,214] [DEBUG] ALLOCATOR: Generating new key pair [2024-04-17 12:16:03,600] [DEBUG] ALLOCATOR: Vagrantfile created. Creating instance. [2024-04-17 12:16:03,600] [INFO] ALLOCATOR: Instance VAGRANT-F72E5B3B-11D7-46DE-A8B3-8557E9D5F6F3 created. [2024-04-17 12:16:03,601] [DEBUG] ALLOCATOR: Vagrantfile created. Creating instance. [2024-04-17 12:16:03,602] [INFO] ALLOCATOR: Instance VAGRANT-2F9CB804-CC23-4972-BADF-4DC8A2403068 created. [2024-04-17 12:16:03,902] [DEBUG] ALLOCATOR: Vagrantfile created. Creating instance. [2024-04-17 12:16:03,902] [INFO] ALLOCATOR: Instance VAGRANT-2504FAA1-F821-46EE-B8A9-86E283308F39 created. [2024-04-17 12:16:59,745] [INFO] ALLOCATOR: Instance VAGRANT-2504FAA1-F821-46EE-B8A9-86E283308F39 started. [2024-04-17 12:17:02,008] [INFO] ALLOCATOR: Inventory file generated at /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml [2024-04-17 12:17:02,887] [INFO] ALLOCATOR: Instance VAGRANT-F72E5B3B-11D7-46DE-A8B3-8557E9D5F6F3 started. [2024-04-17 12:17:04,281] [INFO] ALLOCATOR: Track file generated at /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/track.yaml [2024-04-17 12:17:04,330] [INFO] [744222] [ThreadPoolExecutor-0_1] [workflow_engine]: [allocate-manager-linux-ubuntu-22.04-amd64] Finished task in 64.37 seconds. [2024-04-17 12:17:04,386] [INFO] ALLOCATOR: Instance VAGRANT-2F9CB804-CC23-4972-BADF-4DC8A2403068 started. [2024-04-17 12:17:05,148] [INFO] ALLOCATOR: Inventory file generated at /tmp/dtt1-poc/manager-linux-ubuntu-20.04-amd64/inventory.yaml [2024-04-17 12:17:06,746] [INFO] ALLOCATOR: Inventory file generated at /tmp/dtt1-poc/manager-linux-oracle-9-amd64/inventory.yaml [2024-04-17 12:17:07,390] [INFO] ALLOCATOR: Track file generated at /tmp/dtt1-poc/manager-linux-ubuntu-20.04-amd64/track.yaml [2024-04-17 12:17:07,446] [INFO] [744222] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-manager-linux-ubuntu-20.04-amd64] Finished task in 67.49 seconds. [2024-04-17 12:17:09,054] [INFO] ALLOCATOR: Track file generated at /tmp/dtt1-poc/manager-linux-oracle-9-amd64/track.yaml [2024-04-17 12:17:09,101] [INFO] [744222] [ThreadPoolExecutor-0_2] [workflow_engine]: [allocate-manager-linux-oracle-9-amd64] Finished task in 69.13 seconds. [2024-04-17 12:17:09,116] [INFO] [744222] [ThreadPoolExecutor-0_1] [workflow_engine]: [run-manager-tests] Starting task. [2024-04-17 12:17:09,374] [INFO] TESTER: Running tests for 192.168.57.2 [2024-04-17 12:17:09,375] [INFO] TESTER: Running tests for 192.168.57.2 [2024-04-17 12:17:09,375] [INFO] TESTER: Running tests for 192.168.57.2 [2024-04-17 12:17:09,375] [DEBUG] TESTER: Using extra vars: {'component': 'manager', 'wazuh_version': '4.7.3', 'wazuh_revision': '40714', 'wazuh_branch': None, 'working_dir': '/tmp/tests', 'live': True, 'hosts_ip': ['192.168.57.2', '192.168.57.2', '192.168.57.2'], 'targets': '{wazuh-1: /tmp/dtt1-poc/manager-linux-ubuntu-20.04-amd64/inventory.yaml, wazuh-2: /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml, wazuh-3: /tmp/dtt1-poc/manager-linux-oracle-9-amd64/inventory.yaml}', 'dependencies': '{}', 'local_host_path': '/home/marcelo/wazuh/wazuh-qa/deployability', 'current_user': 'marcelo'} [2024-04-17 12:17:09,377] [DEBUG] TESTER: Rendering template /home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/playbooks/setup.yml [2024-04-17 12:17:09,385] [DEBUG] TESTER: Using inventory: {'all': {'hosts': {'192.168.57.2': {'ansible_port': 22, 'ansible_user': 'vagrant', 'ansible_ssh_private_key_file': '/tmp/wazuh-qa/VAGRANT-F72E5B3B-11D7-46DE-A8B3-8557E9D5F6F3/instance_key'}}}} [2024-04-17 12:17:09,385] [DEBUG] TESTER: Running playbook: [{'hosts': 'localhost', 'become': True, 'become_user': 'marcelo', 'tasks': [{'name': 'Cleaning old key ssh-keygen registries', 'ansible.builtin.command': {'cmd': "ssh-keygen -f /home/marcelo/.ssh/known_hosts -R ''"}, 'loop': ['192.168.57.2', '192.168.57.2', '192.168.57.2']}]}] [2024-04-17 12:17:10,984] [DEBUG] TESTER: Playbook [{'hosts': 'localhost', 'become': True, 'become_user': 'marcelo', 'tasks': [{'name': 'Cleaning old key ssh-keygen registries', 'ansible.builtin.command': {'cmd': "ssh-keygen -f /home/marcelo/.ssh/known_hosts -R ''"}, 'loop': ['192.168.57.2', '192.168.57.2', '192.168.57.2']}]}] finished with status {'skipped': {}, 'ok': {'localhost': 2}, 'dark': {}, 'failures': {}, 'ignored': {}, 'rescued': {}, 'processed': {'localhost': 1}, 'changed': {'localhost': 1}} [2024-04-17 12:17:10,986] [DEBUG] TESTER: Rendering template /home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/playbooks/test.yml [2024-04-17 12:17:10,987] [DEBUG] TESTER: Using inventory: {'all': {'hosts': {'192.168.57.2': {'ansible_port': 22, 'ansible_user': 'vagrant', 'ansible_ssh_private_key_file': '/tmp/wazuh-qa/VAGRANT-F72E5B3B-11D7-46DE-A8B3-8557E9D5F6F3/instance_key'}}}} [2024-04-17 12:17:10,987] [DEBUG] TESTER: Running playbook: [{'hosts': 'localhost', 'become': True, 'become_user': 'marcelo', 'tasks': [{'name': 'Test install for manager', 'command': "python3 -m pytest modules/testing/tests/test_manager/test_install.py -v --wazuh_version=4.7.3 --wazuh_revision=40714 --component=manager --dependencies='{}' --targets='{wazuh-1: /tmp/dtt1-poc/manager-linux-ubuntu-20.04-amd64/inventory.yaml, wazuh-2: /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml, wazuh-3: /tmp/dtt1-poc/manager-linux-oracle-9-amd64/inventory.yaml}' --live=True -s", 'args': {'chdir': '/home/marcelo/wazuh/wazuh-qa/deployability'}}]}] [2024-04-17 12:17:12,591] [INFO] TESTER: Checking connection to ubuntu-20.04 [2024-04-17 12:17:12,853] [ERROR] TESTER: Authentication error. Check SSH credentials in ubuntu-20.04 anager-linux-debian-10-amd64/inventory.yaml" # - wazuh-11: "{working-dir}/manager-linux-debian-11-amd64/inventory.yaml" # - wazuh-12: "{working-dir}/manager-linux-debian-12-amd64/inventory.yaml" - tests: "install,restart,stop,uninstall" - component: "manager" - wazuh-version: "4.7.3" - wazuh-revision: "40714" - live: "True" depends-on: - "allocate-manager-linux-ubuntu-20.04-amd64" - "allocate-manager-linux-ubuntu-22.04-amd64" - "allocate-manager-linux-oracle-9-amd64" ```the testing module logs that the instances have the same IP address, which causes the
sshlogin to fail.workflow.log
``` marcelo@marcelo-B460-AORUS-PRO-AC:~/wazuh/wazuh-qa$ tail -f /tmp/workflow.log [2024-04-17 12:15:59,960] [INFO] [744222] [MainThread] [workflow_engine]: Executing DAG tasks. [2024-04-17 12:15:59,960] [INFO] [744222] [MainThread] [workflow_engine]: Executing tasks in parallel. [2024-04-17 12:15:59,960] [INFO] [744222] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-manager-linux-ubuntu-20.04-amd64] Starting task. [2024-04-17 12:15:59,961] [INFO] [744222] [ThreadPoolExecutor-0_1] [workflow_engine]: [allocate-manager-linux-ubuntu-22.04-amd64] Starting task. [2024-04-17 12:15:59,961] [INFO] [744222] [ThreadPoolExecutor-0_2] [workflow_engine]: [allocate-manager-linux-oracle-9-amd64] Starting task. [2024-04-17 12:16:00,188] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa [2024-04-17 12:16:00,189] [DEBUG] ALLOCATOR: No config provided. Generating from payload [2024-04-17 12:16:00,189] [DEBUG] ALLOCATOR: Generating new key pair [2024-04-17 12:16:00,192] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa [2024-04-17 12:16:00,193] [DEBUG] ALLOCATOR: No config provided. Generating from payload [2024-04-17 12:16:00,193] [DEBUG] ALLOCATOR: Generating new key pair [2024-04-17 12:16:00,214] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa [2024-04-17 12:16:00,214] [DEBUG] ALLOCATOR: No config provided. Generating from payload [2024-04-17 12:16:00,214] [DEBUG] ALLOCATOR: Generating new key pair [2024-04-17 12:16:03,600] [DEBUG] ALLOCATOR: Vagrantfile created. Creating instance. [2024-04-17 12:16:03,600] [INFO] ALLOCATOR: Instance VAGRANT-F72E5B3B-11D7-46DE-A8B3-8557E9D5F6F3 created. [2024-04-17 12:16:03,601] [DEBUG] ALLOCATOR: Vagrantfile created. Creating instance. [2024-04-17 12:16:03,602] [INFO] ALLOCATOR: Instance VAGRANT-2F9CB804-CC23-4972-BADF-4DC8A2403068 created. [2024-04-17 12:16:03,902] [DEBUG] ALLOCATOR: Vagrantfile created. Creating instance. [2024-04-17 12:16:03,902] [INFO] ALLOCATOR: Instance VAGRANT-2504FAA1-F821-46EE-B8A9-86E283308F39 created. [2024-04-17 12:16:59,745] [INFO] ALLOCATOR: Instance VAGRANT-2504FAA1-F821-46EE-B8A9-86E283308F39 started. [2024-04-17 12:17:02,008] [INFO] ALLOCATOR: Inventory file generated at /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml [2024-04-17 12:17:02,887] [INFO] ALLOCATOR: Instance VAGRANT-F72E5B3B-11D7-46DE-A8B3-8557E9D5F6F3 started. [2024-04-17 12:17:04,281] [INFO] ALLOCATOR: Track file generated at /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/track.yaml [2024-04-17 12:17:04,330] [INFO] [744222] [ThreadPoolExecutor-0_1] [workflow_engine]: [allocate-manager-linux-ubuntu-22.04-amd64] Finished task in 64.37 seconds. [2024-04-17 12:17:04,386] [INFO] ALLOCATOR: Instance VAGRANT-2F9CB804-CC23-4972-BADF-4DC8A2403068 started. [2024-04-17 12:17:05,148] [INFO] ALLOCATOR: Inventory file generated at /tmp/dtt1-poc/manager-linux-ubuntu-20.04-amd64/inventory.yaml [2024-04-17 12:17:06,746] [INFO] ALLOCATOR: Inventory file generated at /tmp/dtt1-poc/manager-linux-oracle-9-amd64/inventory.yaml [2024-04-17 12:17:07,390] [INFO] ALLOCATOR: Track file generated at /tmp/dtt1-poc/manager-linux-ubuntu-20.04-amd64/track.yaml [2024-04-17 12:17:07,446] [INFO] [744222] [ThreadPoolExecutor-0_0] [workflow_engine]: [allocate-manager-linux-ubuntu-20.04-amd64] Finished task in 67.49 seconds. [2024-04-17 12:17:09,054] [INFO] ALLOCATOR: Track file generated at /tmp/dtt1-poc/manager-linux-oracle-9-amd64/track.yaml [2024-04-17 12:17:09,101] [INFO] [744222] [ThreadPoolExecutor-0_2] [workflow_engine]: [allocate-manager-linux-oracle-9-amd64] Finished task in 69.13 seconds. [2024-04-17 12:17:09,116] [INFO] [744222] [ThreadPoolExecutor-0_1] [workflow_engine]: [run-manager-tests] Starting task. [2024-04-17 12:17:09,374] [INFO] TESTER: Running tests for 192.168.57.2 [2024-04-17 12:17:09,375] [INFO] TESTER: Running tests for 192.168.57.2 [2024-04-17 12:17:09,375] [INFO] TESTER: Running tests for 192.168.57.2 [2024-04-17 12:17:09,375] [DEBUG] TESTER: Using extra vars: {'component': 'manager', 'wazuh_version': '4.7.3', 'wazuh_revision': '40714', 'wazuh_branch': None, 'working_dir': '/tmp/tests', 'live': True, 'hosts_ip': ['192.168.57.2', '192.168.57.2', '192.168.57.2'], 'targets': '{wazuh-1: /tmp/dtt1-poc/manager-linux-ubuntu-20.04-amd64/inventory.yaml, wazuh-2: /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml, wazuh-3: /tmp/dtt1-poc/manager-linux-oracle-9-amd64/inventory.yaml}', 'dependencies': '{}', 'local_host_path': '/home/marcelo/wazuh/wazuh-qa/deployability', 'current_user': 'marcelo'} [2024-04-17 12:17:09,377] [DEBUG] TESTER: Rendering template /home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/playbooks/setup.yml [2024-04-17 12:17:09,385] [DEBUG] TESTER: Using inventory: {'all': {'hosts': {'192.168.57.2': {'ansible_port': 22, 'ansible_user': 'vagrant', 'ansible_ssh_private_key_file': '/tmp/wazuh-qa/VAGRANT-F72E5B3B-11D7-46DE-A8B3-8557E9D5F6F3/instance_key'}}}} [2024-04-17 12:17:09,385] [DEBUG] TESTER: Running playbook: [{'hosts': 'localhost', 'become': True, 'become_user': 'marcelo', 'tasks': [{'name': 'Cleaning old key ssh-keygen registries', 'ansible.builtin.command': {'cmd': "ssh-keygen -f /home/marcelo/.ssh/known_hosts -R ''"}, 'loop': ['192.168.57.2', '192.168.57.2', '192.168.57.2']}]}] [2024-04-17 12:17:10,984] [DEBUG] TESTER: Playbook [{'hosts': 'localhost', 'become': True, 'become_user': 'marcelo', 'tasks': [{'name': 'Cleaning old key ssh-keygen registries', 'ansible.builtin.command': {'cmd': "ssh-keygen -f /home/marcelo/.ssh/known_hosts -R ''"}, 'loop': ['192.168.57.2', '192.168.57.2', '192.168.57.2']}]}] finished with status {'skipped': {}, 'ok': {'localhost': 2}, 'dark': {}, 'failures': {}, 'ignored': {}, 'rescued': {}, 'processed': {'localhost': 1}, 'changed': {'localhost': 1}} [2024-04-17 12:17:10,986] [DEBUG] TESTER: Rendering template /home/marcelo/wazuh/wazuh-qa/deployability/modules/testing/playbooks/test.yml [2024-04-17 12:17:10,987] [DEBUG] TESTER: Using inventory: {'all': {'hosts': {'192.168.57.2': {'ansible_port': 22, 'ansible_user': 'vagrant', 'ansible_ssh_private_key_file': '/tmp/wazuh-qa/VAGRANT-F72E5B3B-11D7-46DE-A8B3-8557E9D5F6F3/instance_key'}}}} [2024-04-17 12:17:10,987] [DEBUG] TESTER: Running playbook: [{'hosts': 'localhost', 'become': True, 'become_user': 'marcelo', 'tasks': [{'name': 'Test install for manager', 'command': "python3 -m pytest modules/testing/tests/test_manager/test_install.py -v --wazuh_version=4.7.3 --wazuh_revision=40714 --component=manager --dependencies='{}' --targets='{wazuh-1: /tmp/dtt1-poc/manager-linux-ubuntu-20.04-amd64/inventory.yaml, wazuh-2: /tmp/dtt1-poc/manager-linux-ubuntu-22.04-amd64/inventory.yaml, wazuh-3: /tmp/dtt1-poc/manager-linux-oracle-9-amd64/inventory.yaml}' --live=True -s", 'args': {'chdir': '/home/marcelo/wazuh/wazuh-qa/deployability'}}]}] [2024-04-17 12:17:12,591] [INFO] TESTER: Checking connection to ubuntu-20.04 [2024-04-17 12:17:12,853] [ERROR] TESTER: Authentication error. Check SSH credentials in ubuntu-20.04 ```These are the inventory files generated by the allocator:
inventory.zip
All of them have the same IP address.