search

wazuh / wazuh-qa

Wazuh - Quality Assurance
GNU General Public License v2.0
61 stars 30 forks source link

Validation of system dependencies is added to the Allocation module #5336

Closed c-bordon closed 2 weeks ago

c-bordon commented 2 weeks ago

Close https://github.com/wazuh/wazuh-qa/issues/5295

Validation of system dependencies is added for the correct functioning of the Allocation module. With this we validate that the packages that are necessary for operation are installed prior to attempting to create the VMs. With this, errors were discovered when trying to run the module on a Centos 7, here the error was in the python dependencies.

Test Ubuntu 22.04

vagrant@ubuntu-jammy:~$ git clone https://github.com/wazuh/wazuh-qa.git
Cloning into 'wazuh-qa'...
remote: Enumerating objects: 105176, done.
remote: Counting objects: 100% (7687/7687), done.
remote: Compressing objects: 100% (2581/2581), done.
remote: Total 105176 (delta 5045), reused 7324 (delta 4773), pack-reused 97489
Receiving objects: 100% (105176/105176), 50.23 MiB | 8.17 MiB/s, done.
Resolving deltas: 100% (66414/66414), done.
vagrant@ubuntu-jammy:~$ cd wazuh-qa/
vagrant@ubuntu-jammy:~/wazuh-qa$ git checkout bug/5295-dtt1-allocator-module-the-allocation-fails-if-the-sshpass-utility-is-not-installed
Branch 'bug/5295-dtt1-allocator-module-the-allocation-fails-if-the-sshpass-utility-is-not-installed' set up to track remote branch 'bug/5295-dtt1-allocator-module-the-allocation-fails-if-the-sshpass-utility-is-not-installed' from 'origin'.
Switched to a new branch 'bug/5295-dtt1-allocator-module-the-allocation-fails-if-the-sshpass-utility-is-not-installed'
vagrant@ubuntu-jammy:~/wazuh-qa$ git pull
Already up to date.
vagrant@ubuntu-jammy:~/wazuh-qa$ cat /etc/*release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
vagrant@ubuntu-jammy:~/wazuh-qa$ python3 deployability/modules/allocation/main.py --provider vagrant --size small --composite-name linux-centos-7-amd64
[2024-05-03 14:47:24] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'
[2024-05-03 14:47:24] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'
[2024-05-03 14:47:24] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
Traceback (most recent call last):
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 39, in <module>
    main()
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 35, in main
    Allocator.run(InputPayload(**vars(parse_arguments())))
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 37, in run
    return cls.__create(payload)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 56, in __create
    instance = provider.create_instance(
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/generic/provider.py", line 70, in create_instance
    return cls._create_instance(base_dir, params, config, ssh_key)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 48, in _create_instance
    cls.validate_dependencies(params.composite_name)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 453, in validate_dependencies
    raise ValueError(f"Missing package: {package}")
ValueError: Missing package: vagrant
vagrant@ubuntu-jammy:~/wazuh-qa$ python3 deployability/modules/allocation/main.py --provider vagrant --size small --composite-name linux-centos-7-amd64
[2024-05-03 14:51:49] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'
[2024-05-03 14:51:49] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'
[2024-05-03 14:51:49] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
Traceback (most recent call last):
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 39, in <module>
    main()
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 35, in main
    Allocator.run(InputPayload(**vars(parse_arguments())))
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 37, in run
    return cls.__create(payload)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 56, in __create
    instance = provider.create_instance(
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/generic/provider.py", line 70, in create_instance
    return cls._create_instance(base_dir, params, config, ssh_key)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 48, in _create_instance
    cls.validate_dependencies(params.composite_name)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 451, in validate_dependencies
    raise ValueError(f"Missing package: {package}")
ValueError: Missing package: virtualbox
sudo ./aws/install^C
vagrant@ubuntu-jammy:~/wazuh-qa$ python3 deployability/modules/allocation/main.py --provider vagrant --size small --composite-name macos-sonoma-14.0-arm64
[2024-05-03 15:15:46] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'
[2024-05-03 15:15:46] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'
[2024-05-03 15:15:46] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
Traceback (most recent call last):
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 39, in <module>
    main()
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 35, in main
    Allocator.run(InputPayload(**vars(parse_arguments())))
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 37, in run
    return cls.__create(payload)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 56, in __create
    instance = provider.create_instance(
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/generic/provider.py", line 70, in create_instance
    return cls._create_instance(base_dir, params, config, ssh_key)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 48, in _create_instance
    cls.validate_dependencies(params.composite_name)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 448, in validate_dependencies
    raise ValueError(f"Missing package: {package}")
ValueError: Missing package: sshpass
vagrant@ubuntu-jammy:~/wazuh-qa$ python3 deployability/modules/allocation/main.py --provider vagrant --size small --composite-name macos-sonoma-14.0-arm64
[2024-05-03 15:19:45] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'
[2024-05-03 15:19:45] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'
[2024-05-03 15:19:45] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
Traceback (most recent call last):
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 39, in <module>
    main()
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 35, in main
    Allocator.run(InputPayload(**vars(parse_arguments())))
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 37, in run
    return cls.__create(payload)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 56, in __create
    instance = provider.create_instance(
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/generic/provider.py", line 70, in create_instance
    return cls._create_instance(base_dir, params, config, ssh_key)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 48, in _create_instance
    cls.validate_dependencies(params.composite_name)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 450, in validate_dependencies
    raise ValueError(f"Missing package: {package}")
ValueError: Missing package: awscli
c-bordon commented 2 weeks ago

New tests after PR review:

vagrant@ubuntu-jammy:~/wazuh-qa$ python3 deployability/modules/allocation/main.py --provider vagrant --size small --composite-name linux-ubuntu-22.04-amd64
[2024-05-03 18:06:01] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'
[2024-05-03 18:06:01] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'
[2024-05-03 18:06:01] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
Traceback (most recent call last):
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 39, in <module>
    main()
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 35, in main
    Allocator.run(InputPayload(**vars(parse_arguments())))
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 37, in run
    return cls.__create(payload)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 56, in __create
    instance = provider.create_instance(
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/generic/provider.py", line 70, in create_instance
    return cls._create_instance(base_dir, params, config, ssh_key)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 48, in _create_instance
    cls.validate_dependencies(params.composite_name)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 445, in validate_dependencies
    raise ValueError(f"Missing package: {missing_dependecy}")
ValueError: Missing package: virtualbox
vagrant@ubuntu-jammy:~/wazuh-qa$ python3 deployability/modules/allocation/main.py --provider vagrant --size small --composite-name linux-ubuntu-22.04-amd64
[2024-05-03 18:06:41] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'
[2024-05-03 18:06:41] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'
[2024-05-03 18:06:41] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
Traceback (most recent call last):
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 39, in <module>
    main()
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 35, in main
    Allocator.run(InputPayload(**vars(parse_arguments())))
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 37, in run
    return cls.__create(payload)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 56, in __create
    instance = provider.create_instance(
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/generic/provider.py", line 70, in create_instance
    return cls._create_instance(base_dir, params, config, ssh_key)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 48, in _create_instance
    cls.validate_dependencies(params.composite_name)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 445, in validate_dependencies
    raise ValueError(f"Missing package: {missing_dependecy}")
ValueError: Missing package: vagrant
vagrant@ubuntu-jammy:~/wazuh-qa$ python3 deployability/modules/allocation/main.py --provider vagrant --size small --composite-name macos-sonoma-14.0-arm64
[2024-05-03 18:07:07] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'
[2024-05-03 18:07:07] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'
[2024-05-03 18:07:07] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
Traceback (most recent call last):
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 39, in <module>
    main()
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 35, in main
    Allocator.run(InputPayload(**vars(parse_arguments())))
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 37, in run
    return cls.__create(payload)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 56, in __create
    instance = provider.create_instance(
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/generic/provider.py", line 70, in create_instance
    return cls._create_instance(base_dir, params, config, ssh_key)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 48, in _create_instance
    cls.validate_dependencies(params.composite_name)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 445, in validate_dependencies
    raise ValueError(f"Missing package: {missing_dependecy}")
ValueError: Missing package: sshpass
vagrant@ubuntu-jammy:~/wazuh-qa$ python3 deployability/modules/allocation/main.py --provider vagrant --size small --composite-name macos-sonoma-14.0-arm64
[2024-05-03 18:17:22] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'
[2024-05-03 18:17:22] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'
[2024-05-03 18:17:22] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
Traceback (most recent call last):
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 39, in <module>
    main()
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 35, in main
    Allocator.run(InputPayload(**vars(parse_arguments())))
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 37, in run
    return cls.__create(payload)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 56, in __create
    instance = provider.create_instance(
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/generic/provider.py", line 70, in create_instance
    return cls._create_instance(base_dir, params, config, ssh_key)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 48, in _create_instance
    cls.validate_dependencies(params.composite_name)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 445, in validate_dependencies
    raise ValueError(f"Missing package: {missing_dependecy}")
ValueError: Missing package: awscli
c-bordon commented 2 weeks ago

Improved missing dependencies message

vagrant@ubuntu-jammy:~/wazuh-qa$ python3 deployability/modules/allocation/main.py --provider vagrant --size small --composite-name linux-ubuntu-22.04-amd64
[2024-05-03 18:41:19] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'
[2024-05-03 18:41:19] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'
[2024-05-03 18:41:19] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
Traceback (most recent call last):
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 39, in <module>
    main()
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 35, in main
    Allocator.run(InputPayload(**vars(parse_arguments())))
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 37, in run
    return cls.__create(payload)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 56, in __create
    instance = provider.create_instance(
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/generic/provider.py", line 70, in create_instance
    return cls._create_instance(base_dir, params, config, ssh_key)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 48, in _create_instance
    cls.validate_dependencies(params.composite_name)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 449, in validate_dependencies
    raise ValueError(f"Missing dependencies: {missing_dependencies}")
ValueError: Missing dependencies: ['vagrant', 'virtualbox']
vagrant@ubuntu-jammy:~/wazuh-qa$ python3 deployability/modules/allocation/main.py --provider vagrant --size small --composite-name macos-sonoma-14.0-arm64
[2024-05-03 18:41:37] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'
[2024-05-03 18:41:37] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'
[2024-05-03 18:41:37] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
Traceback (most recent call last):
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 39, in <module>
    main()
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 35, in main
    Allocator.run(InputPayload(**vars(parse_arguments())))
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 37, in run
    return cls.__create(payload)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 56, in __create
    instance = provider.create_instance(
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/generic/provider.py", line 70, in create_instance
    return cls._create_instance(base_dir, params, config, ssh_key)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 48, in _create_instance
    cls.validate_dependencies(params.composite_name)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 449, in validate_dependencies
    raise ValueError(f"Missing dependencies: {missing_dependencies}")
ValueError: Missing dependencies: ['sshpass', 'awscli']
vagrant@ubuntu-jammy:~/wazuh-qa$ python3 deployability/modules/allocation/main.py --provider vagrant --size small --composite-name macos-sonoma-14.0-arm64
[2024-05-03 18:42:13] [DEBUG] SPNEGO._GSS: Python gssapi not available, cannot use any GSSAPIProxy protocols: No module named 'gssapi'
[2024-05-03 18:42:13] [DEBUG] SPNEGO._GSS: Python gssapi IOV extension not available: No module named 'gssapi'
[2024-05-03 18:42:13] [INFO] ALLOCATOR: Creating instance at /tmp/wazuh-qa
Traceback (most recent call last):
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 39, in <module>
    main()
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/main.py", line 35, in main
    Allocator.run(InputPayload(**vars(parse_arguments())))
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 37, in run
    return cls.__create(payload)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/allocation.py", line 56, in __create
    instance = provider.create_instance(
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/generic/provider.py", line 70, in create_instance
    return cls._create_instance(base_dir, params, config, ssh_key)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 48, in _create_instance
    cls.validate_dependencies(params.composite_name)
  File "/home/vagrant/wazuh-qa/deployability/modules/allocation/vagrant/provider.py", line 447, in validate_dependencies
    raise ValueError(f"Missing dependency: {missing_dependencies[0]}")
ValueError: Missing dependency: awscli