search

wazuh / wazuh-installation-assistant

Wazuh - Installation assistant
https://wazuh.com/
GNU General Public License v2.0
0 stars 1 forks source link

Installation assistant fails to remove wazuh component upon lack of space error #5

Open Rebits opened 4 months ago

Rebits commented 4 months ago
Wazuh version Install type Action performed Platform
4.8.0-beta5 Wazuh installation assistant Install Ubuntu 22

Description

It has been detected that in case of failing the installation of the wazuh-dashboard due to lack of space in the system, the wazuh component previously installed are not removed correctly

Steps to reproduce

root@ip-172-31-44-9:/home/ubuntu# apt list --installed | grep wazuh-manager
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

wazuh-manager/now 4.8.0-1 amd64 [installed,local]
root@ip-172-31-44-9:/home/ubuntu# apt list --installed | grep wazuh-indexer

wazuh-indexer/now 4.8.0-1 amd64 [installed,local]

root@ip-172-31-44-9:/home/ubuntu# ls /var/ossec
ls: cannot access '/var/ossec': No such file or directory

root@ip-172-31-44-9:/home/ubuntu# ls /etc | grep wazuh

Evidence

Installation log - First installation ``` root@ip-172-31-44-9:/home/ubuntu# curl -sO https://packages-dev.wazuh.com/4.8/wazuh-install.sh && sudo bash ./wazuh-install.sh -a 12/04/2024 08:46:28 INFO: Starting Wazuh installation assistant. Wazuh version: 4.8.0 12/04/2024 08:46:28 INFO: Verbose logging redirected to /var/log/wazuh-install.log 12/04/2024 08:46:38 INFO: Verifying that your system meets the recommended minimum hardware requirements. 12/04/2024 08:46:38 INFO: Wazuh web interface port will be 443. 12/04/2024 08:46:44 INFO: --- Dependencies ---- 12/04/2024 08:46:44 INFO: Installing apt-transport-https. 12/04/2024 08:46:54 INFO: Wazuh development repository added. 12/04/2024 08:46:54 INFO: --- Configuration files --- 12/04/2024 08:46:54 INFO: Generating configuration files. 12/04/2024 08:46:54 INFO: Generating the root certificate. 12/04/2024 08:46:54 INFO: Generating Admin certificates. 12/04/2024 08:46:55 INFO: Generating Wazuh indexer certificates. 12/04/2024 08:46:56 INFO: Generating Filebeat certificates. 12/04/2024 08:46:56 INFO: Generating Wazuh dashboard certificates. 12/04/2024 08:46:57 INFO: Created wazuh-install-files.tar. It contains the Wazuh cluster key, certificates, and passwords necessary for installation. 12/04/2024 08:46:57 INFO: --- Wazuh indexer --- 12/04/2024 08:46:57 INFO: Starting Wazuh indexer installation. 12/04/2024 08:48:36 INFO: Wazuh indexer installation finished. 12/04/2024 08:48:36 INFO: Wazuh indexer post-install configuration finished. 12/04/2024 08:48:36 INFO: Starting service wazuh-indexer. 12/04/2024 08:49:01 INFO: wazuh-indexer service started. 12/04/2024 08:49:01 INFO: Initializing Wazuh indexer cluster security settings. 12/04/2024 08:49:12 INFO: Wazuh indexer cluster security configuration initialized. 12/04/2024 08:49:12 INFO: Wazuh indexer cluster initialized. 12/04/2024 08:49:12 INFO: --- Wazuh server --- 12/04/2024 08:49:12 INFO: Starting the Wazuh manager installation. 12/04/2024 08:50:46 INFO: Wazuh manager installation finished. 12/04/2024 08:50:46 INFO: Wazuh manager vulnerability detection configuration finished. 12/04/2024 08:50:46 INFO: Starting service wazuh-manager. 12/04/2024 08:51:07 INFO: wazuh-manager service started. 12/04/2024 08:51:07 INFO: Starting Filebeat installation. 12/04/2024 08:51:22 INFO: Filebeat installation finished. 12/04/2024 08:51:24 INFO: Filebeat post-install configuration finished. 12/04/2024 08:51:24 INFO: Starting service filebeat. 12/04/2024 08:51:26 INFO: filebeat service started. 12/04/2024 08:51:26 INFO: --- Wazuh dashboard --- 12/04/2024 08:51:26 INFO: Starting Wazuh dashboard installation. 12/04/2024 08:51:36 ERROR: Wazuh dashboard installation failed. 12/04/2024 08:51:36 INFO: --- Removing existing Wazuh installation --- 12/04/2024 08:51:39 INFO: Installation cleaned. Check the /var/log/wazuh-install.log file to learn more about the issue. ```
/var/log/wazuh-install.log ``` root@ip-172-31-44-9:/home/ubuntu# cat /var/log/wazuh-install.log^C root@ip-172-31-44-9:/home/ubuntu# cat /var/log/wazuh-install.log 12/04/2024 08:46:28 INFO: Starting Wazuh installation assistant. Wazuh version: 4.8.0 12/04/2024 08:46:28 INFO: Verbose logging redirected to /var/log/wazuh-install.log Hit:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy InRelease Hit:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates InRelease Hit:3 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-backports InRelease Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease Reading package lists... 12/04/2024 08:46:38 INFO: Verifying that your system meets the recommended minimum hardware requirements. 12/04/2024 08:46:38 INFO: Wazuh web interface port will be 443. Hit:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy InRelease Hit:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates InRelease Hit:3 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-backports InRelease Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease Reading package lists... 12/04/2024 08:46:44 INFO: --- Dependencies ---- 12/04/2024 08:46:44 INFO: Installing apt-transport-https. Reading package lists... Building dependency tree... Reading state information... The following NEW packages will be installed: apt-transport-https 0 upgraded, 1 newly installed, 0 to remove and 43 not upgraded. Need to get 1,510 B of archives. After this operation, 170 kB of additional disk space will be used. Get:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates/ NEEDRESTART-VER: 3.5 NEEDRESTART-KCUR: 6.5.0-1014-aws NEEDRESTART-KEXP: 6.5.0-1014-aws NEEDRESTART-KSTA: 1sly unselected package apt-transport-https. gpg: keyring '/usr/share/keyrings/wazuh.gpg' created gpg: directory '/root/.gnupg' created gpg: /root/.gnupg/trustdb.gpg: trustdb created gpg: key 96B3EE5F29111145: public key "Wazuh.com (Wazuh Signing Key) " imported gpg: Total number processed: 1 gpg: imported: 1 deb [signed-by=/usr/share/keyrings/wazuh.gpg] https://packages-dev.wazuh.com/pre-release/apt/ unstable main Hit:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy InRelease Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease Hit:3 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-updates InRelease Hit:4 http://us-east-1.ec2.archive.ubuntu.com/ubuntu jammy-backports InRelease Get:5 https://packages-dev.wazuh.com/pre-release/apt unstable InRelease [17.3 kB] Get:6 https://packages-dev.wazuh.com/pre-release/apt unstable/main amd64 Packages [37.1 kB] Fetched 54.4 kB in 1s (41.8 kB/s) Reading package lists... 12/04/2024 08:46:54 INFO: Wazuh development repository added. 12/04/2024 08:46:54 INFO: --- Configuration files --- 12/04/2024 08:46:54 INFO: Generating configuration files. 12/04/2024 08:46:54 INFO: Generating the root certificate. 12/04/2024 08:46:54 INFO: Generating Admin certificates. 12/04/2024 08:46:55 INFO: Generating Wazuh indexer certificates. 12/04/2024 08:46:56 INFO: Generating Filebeat certificates. 12/04/2024 08:46:56 INFO: Generating Wazuh dashboard certificates. 12/04/2024 08:46:57 INFO: Created wazuh-install-files.tar. It contains the Wazuh cluster key, certificates, and passwords necessary for installation. 12/04/2024 08:46:57 INFO: --- Wazuh indexer --- 12/04/2024 08:46:57 INFO: Starting Wazuh indexer installation. Reading package lists... Building dependency tree... Reading state information... The following NEW packages will be installed: wazuh-indexer 0 upgraded, 1 newly installed, 0 to remove and 43 not upgraded. Need to get 757 MB of archives. After this operation, 1,050 MB of additional disk space will be used. Get:1 https://packages-dev.wazuh.com/pre-release/apt unstable/main amd64 NEEDRESTART-VER: 3.5 NEEDRESTART-KCUR: 6.5.0-1014-aws NEEDRESTART-KEXP: 6.5.0-1014-aws NEEDRESTART-KSTA: 1kage wazuh-indexer. 12/04/2024 08:48:36 INFO: Wazuh indexer installation finished. 12/04/2024 08:48:36 INFO: Wazuh indexer post-install configuration finished. 12/04/2024 08:48:36 INFO: Starting service wazuh-indexer. Created symlink /etc/systemd/system/multi-user.target.wants/wazuh-indexer.service → /lib/systemd/system/wazuh-indexer.service. 12/04/2024 08:49:01 INFO: wazuh-indexer service started. 12/04/2024 08:49:01 INFO: Initializing Wazuh indexer cluster security settings. ************************************************************************** ** This tool will be deprecated in the next major release of OpenSearch ** ** https://github.com/opensearch-project/security/issues/1755 ** ************************************************************************** Security Admin v7 Will connect to 127.0.0.1:9200 ... done Connected as "CN=admin,OU=Wazuh,O=Wazuh,L=California,C=US" OpenSearch Version: 2.10.0 Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ... Clustername: wazuh-cluster Clusterstate: GREEN Number of nodes: 1 Number of data nodes: 1 .opendistro_security index does not exists, attempt to create it ... done (0-all replicas) Populate config from /etc/wazuh-indexer/opensearch-security/ Will update '/config' with /etc/wazuh-indexer/opensearch-security/config.yml SUCC: Configuration for 'config' created or updated Will update '/roles' with /etc/wazuh-indexer/opensearch-security/roles.yml SUCC: Configuration for 'roles' created or updated Will update '/rolesmapping' with /etc/wazuh-indexer/opensearch-security/roles_mapping.yml SUCC: Configuration for 'rolesmapping' created or updated Will update '/internalusers' with /etc/wazuh-indexer/opensearch-security/internal_users.yml SUCC: Configuration for 'internalusers' created or updated Will update '/actiongroups' with /etc/wazuh-indexer/opensearch-security/action_groups.yml SUCC: Configuration for 'actiongroups' created or updated Will update '/tenants' with /etc/wazuh-indexer/opensearch-security/tenants.yml SUCC: Configuration for 'tenants' created or updated Will update '/nodesdn' with /etc/wazuh-indexer/opensearch-security/nodes_dn.yml SUCC: Configuration for 'nodesdn' created or updated Will update '/whitelist' with /etc/wazuh-indexer/opensearch-security/whitelist.yml SUCC: Configuration for 'whitelist' created or updated Will update '/audit' with /etc/wazuh-indexer/opensearch-security/audit.yml SUCC: Configuration for 'audit' created or updated Will update '/allowlist' with /etc/wazuh-indexer/opensearch-security/allowlist.yml SUCC: Configuration for 'allowlist' created or updated SUCC: Expected 10 config types for node {"updated_config_types":["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"],"updated_config_size":10,"message":null} is 10 (["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"]) due to: null Done with success 12/04/2024 08:49:12 INFO: Wazuh indexer cluster security configuration initialized. 12/04/2024 08:49:12 INFO: Wazuh indexer cluster initialized. 12/04/2024 08:49:12 INFO: --- Wazuh server --- 12/04/2024 08:49:12 INFO: Starting the Wazuh manager installation. Reading package lists... Building dependency tree... Reading state information... Suggested packages: expect The following NEW packages will be installed: wazuh-manager 0 upgraded, 1 newly installed, 0 to remove and 43 not upgraded. Need to get 311 MB of archives. After this operation, 914 MB of additional disk space will be used. Get:1 https://packages-dev.wazuh.com/pre-releas NEEDRESTART-VER: 3.5 NEEDRESTART-KCUR: 6.5.0-1014-aws NEEDRESTART-KEXP: 6.5.0-1014-aws NEEDRESTART-KSTA: 1reviously unselected package wazuh-manager. 12/04/2024 08:50:46 INFO: Wazuh manager installation finished. 12/04/2024 08:50:46 INFO: Wazuh manager vulnerability detection configuration finished. 12/04/2024 08:50:46 INFO: Starting service wazuh-manager. Created symlink /etc/systemd/system/multi-user.target.wants/wazuh-manager.service → /lib/systemd/system/wazuh-manager.service. 12/04/2024 08:51:07 INFO: wazuh-manager service started. 12/04/2024 08:51:07 INFO: Starting Filebeat installation. Reading package lists... Building dependency tree... Reading state information... The following NEW packages will be installed: filebeat 0 upgraded, 1 newly installed, 0 to remove and 43 not upgraded. Need to get 22.1 MB of archives. After this operation, 73.6 MB of additional disk space will be used. Get:1 https://packages-dev.wazuh.com/pre-release/apt unstable/main amd64 file NEEDRESTART-VER: 3.5 NEEDRESTART-KCUR: 6.5.0-1014-aws NEEDRESTART-KEXP: 6.5.0-1014-aws NEEDRESTART-KSTA: 1eat. 12/04/2024 08:51:22 INFO: Filebeat installation finished. wazuh/ wazuh/_meta/ wazuh/_meta/docs.asciidoc wazuh/_meta/fields.yml wazuh/_meta/config.yml wazuh/alerts/ wazuh/alerts/config/ wazuh/alerts/config/alerts.yml wazuh/alerts/manifest.yml wazuh/alerts/ingest/ wazuh/alerts/ingest/pipeline.json wazuh/module.yml wazuh/archives/ wazuh/archives/config/ wazuh/archives/config/archives.yml wazuh/archives/manifest.yml wazuh/archives/ingest/ wazuh/archives/ingest/pipeline.json Created filebeat keystore Successfully updated the keystore Successfully updated the keystore 12/04/2024 08:51:24 INFO: Filebeat post-install configuration finished. 12/04/2024 08:51:24 INFO: Starting service filebeat. Synchronizing state of filebeat.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable filebeat Created symlink /etc/systemd/system/multi-user.target.wants/filebeat.service → /lib/systemd/system/filebeat.service. 12/04/2024 08:51:26 INFO: filebeat service started. 12/04/2024 08:51:26 INFO: --- Wazuh dashboard --- 12/04/2024 08:51:26 INFO: Starting Wazuh dashboard installation. Reading package lists... Building dependency tree... Reading state information... The following NEW packages will be installed: wazuh-dashboard 0 upgraded, 1 newly installed, 0 to remove and 43 not upgraded. Need to get 186 MB of archives. After this operation, 988 MB of additional disk space will be used. Get:1 https://packages-dev.wazuh.com/pre-release/apt unstable/main amd64 E: Sub-process /usr/bin/dpkg returned an error code (2)ut 'bolt' to '/var/lib/dpkg/status': No space left on devicetes 12/04/2024 08:51:36 ERROR: Wazuh dashboard installation failed. 12/04/2024 08:51:36 INFO: --- Removing existing Wazuh installation --- 12/04/2024 08:51:39 INFO: Installation cleaned. Check the /var/log/wazuh-install.log file to learn more about the issue. root@ip-172-31-44-9:/home/ubuntu# df -h Filesystem Size Used Avail Use% Mounted on /dev/root 7.6G 4.2G 3.4G 56% / tmpfs 3.9G 516K 3.9G 1% /dev/shm tmpfs 1.6G 872K 1.6G 1% /run tmpfs 5.0M 0 5.0M 0% /run/lock efivarfs 128K 3.6K 120K 3% /sys/firmware/efi/efivars /dev/nvme0n1p15 105M 6.1M 99M 6% /boot/efi tmpfs 784M 4.0K 784M 1% /run/user/1000 ```
c-bordon commented 3 months ago

Update report

I was able to replicate the error, apparently the problem is in the handling of the validation of whether wazuh-manager or wazuh-indexer are uninstalled.

21/05/2024 18:54:48 INFO: --- Removing existing Wazuh installation ---
21/05/2024 18:54:48 INFO: Removing Wazuh manager.
Reading package lists...
E: Write error - write (28: No space left on device)
E: IO Error saving source cache
E: The package lists or status file could not be parsed or opened.
21/05/2024 18:54:49 INFO: Wazuh manager removed.

In turn, the uninstallation process was incomplete, causing problems when trying to manually uninstall the product:

ubuntu@ip-172-31-86-216:~$ sudo apt purge -y wazuh-manager
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
  wazuh-manager*
0 upgraded, 0 newly installed, 1 to remove and 195 not upgraded.
After this operation, 916 MB disk space will be freed.
(Reading database ... 86331 files and directories currently installed.)
Removing wazuh-manager (4.8.0-1) ...
dpkg: error processing package wazuh-manager (--remove):
 installed wazuh-manager package pre-removal script subprocess returned error exit status 127
dpkg: too many errors, stopping
Errors were encountered while processing:
 wazuh-manager
Processing was halted because there were too many errors.
E: Sub-process /usr/bin/dpkg returned an error code (1)
ubuntu@ip-172-31-86-216:~$ apt list --installed | grep wazuh-manager

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

wazuh-manager/now 4.8.0-1 amd64 [installed,local]

If the uninstall is truncated, you cannot retry the uninstall, perhaps the best solution is to validate available storage before installing

davidjiglesias commented 2 months ago

We should understand the needed space to install the different Wazuh components. The assistant must enforce the needed space is available prior to installation.

c-bordon commented 1 month ago

For this issue, we need to investigate:

The installation assistant must be able to identify the storage required at each installation point since the user may have different disks mounted in the different directories. We must be able to perform the prior analysis, according to the type of installation (AIO or distributed) and by component.

Enaraque commented 2 weeks ago

This issue goes on hold due to higher priority tasks

Enaraque commented 1 day ago

Update report

The main directories used in the installation of Wazuh components have been investigated.

The affected directories are common to all. These are:

In addition, the Wazuh manager makes use of /var/ossec/.

Once we have figured out which directories each component uses, we have to estimate a used size of each one. To do this, we will download an AIO with both package managers (APT and RPM) and get the total size taken by each component in those directories.

Next approach

Get the total size taken by each component in each directory. After obtaining the necessary information, we will start developing the code logic.