Closed 72nomada closed 10 months ago
After considering adding support for a single-node installation (AIO) and distributed-node installation, I noticed that maybe, it is not necessary to add too much code to automatize the installation process of the offline installation.
With some investigation, I noticed that what we are trying to do is the same that the WIA can do currently (AIO and install the Wazuh central components separately) but instead of using the packages manager to install the packages from the repositories, it would use the packages managers to perform local installations, using the packages contained in the tar file resulting of the -dw
option of the WIA.
We do not only have to consider the packages, but also other resources, such as certificates, the Filebeat module, the Wazuh GPG key, the Wazuh Filebeat template, and the Filebeat package.
It is necessary to consider the structure of the resulting wazuh-offline
directory of the wazuh-offline.tar.gz
file:
root@ubuntu22:/home/vagrant# tree wazuh-offline
wazuh-offline
├── wazuh-files
│ ├── filebeat.yml
│ ├── GPG-KEY-WAZUH
│ ├── wazuh-filebeat-0.2.tar.gz
│ └── wazuh-template.json
└── wazuh-packages
├── filebeat-oss-7.10.2-amd64.deb
├── wazuh-dashboard_4.6.0-1_amd64.deb
├── wazuh-indexer_4.6.0-1_amd64.deb
└── wazuh-manager_4.6.0-1_amd64.deb
2 directories, 8 files
Considering the size of the issue, it is nearly compulsory to divide it into different tasks, following order to perform the correct development:
-of
for example), and this option should work with options -a
, -wi
, -ws
, -wd
Besides, it is necessary to:
After this, some validations must be done to ensure the correct behavior:
Created some EC2 instances to test the development in other OS. This issue goes to On Hold due to https://github.com/wazuh/wazuh-indexer/issues/79
Currently I am working on some errors related to the RPM-based systems. It seems that, with no internet connection, the yum install <package>
can not be performed because the command tries to validate the GPG key.
+ eval 'yum install /home/ec2-user/wazuh-offline/wazuh-packages/wazuh-indexer-4.8.0-1.x86_64.rpm -y 2>&1'
+++ yum install /home/ec2-user/wazuh-offline/wazuh-packages/wazuh-indexer-4.8.0-1.x86_64.rpm -y
+ yum_output='Updating Subscription Management repositories.
Unable to read consumer identity
This system is not registered with an entitlement server. You can use subscription-manager to register.
EL-9 - Wazuh 0.0 B/s | 0 B 01:52
Errors during downloading metadata for repository '\''wazuh'\'':
- Curl error (28): Timeout was reached for https://packages-dev.wazuh.com/pre-release/yum/repodata/repomd.xml [Failed to connect to packages-dev.wazuh.com port 443: Connection timed out]
Error: Failed to download metadata for repo '\''wazuh'\'': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried'
+ install_result=1
+ eval 'echo ${yum_output} 2>&1 | tee -a /var/log/wazuh-install.log'
++ tee -a /var/log/wazuh-install.log
++ echo Updating Subscription Management repositories. Unable to read consumer identity This system is not registered with an entitlement server. You can use subscription-manager to register. EL-9 - Wazuh 0.0 B/s '|' 0 B 01:52 Errors during downloading metadata for repository ''\''wazuh'\'':' - Curl error '(28):' Timeout was reached for https://packages-dev.wazuh.com/pre-release/yum/repodata/repomd.xml '[Failed' to connect to packages-dev.wazuh.com port 443: Connection timed 'out]' Error: Failed to download metadata for repo ''\''wazuh'\'':' Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
Updating Subscription Management repositories. Unable to read consumer identity This system is not registered with an entitlement server. You can use subscription-manager to register. EL-9 - Wazuh 0.0 B/s | 0 B 01:52 Errors during downloading metadata for repository 'wazuh': - Curl error (28): Timeout was reached for https://packages-dev.wazuh.com/pre-release/yum/repodata/repomd.xml [Failed to connect to packages-dev.wazuh.com port 443: Connection timed out] Error: Failed to download metadata for repo 'wazuh': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
+ install_result=1
But, if I run this command manually, no errors are generated:
[root@ip-172-31-36-51 ec2-user]# eval 'yum install /home/ec2-user/wazuh-offline/wazuh-packages/wazuh-indexer-4.8.0-1.x86_64.rpm -y 2>&1'
Updating Subscription Management repositories.
Unable to read consumer identity
This system is not registered with an entitlement server. You can use subscription-manager to register.
Last metadata expiration check: 1:39:15 ago on Wed 13 Dec 2023 04:57:26 PM UTC.
Dependencies resolved.
===================================================
Package Arch Version Repository Size
===================================================
Installing:
wazuh-indexer x86_64 4.8.0-1 @commandline 743 M
Transaction Summary
===================================================
Install 1 Package
Total size: 743 M
Installed size: 1.0 G
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Running scriptlet: wazuh-indexer-4.8.0-1.x 1/1
Installing : wazuh-indexer-4.8.0-1.x 1/1
Running scriptlet: wazuh-indexer-4.8.0-1.x 1/1
Created opensearch keystore in /etc/wazuh-indexer/opensearch.keystore
Verifying : wazuh-indexer-4.8.0-1.x 1/1
Installed products updated.
Installed:
wazuh-indexer-4.8.0-1.x86_64
Complete!
[root@ip-172-31-36-51 ec2-user]#
It is necessary to study the root cause of this situation and fix it. This behavior is seen in RHEL9 and CentOS8. In DEB-based systems, this behavior is not reproduced.
:heavy_check_mark: The proposed solution is to use rpm
instead of yum
in offline installation. The problem is that yum
as the package manager tries to find the dependencies of the Wazuh packages and update the database. In the offline installation, this is not necessary, so rpm
can be used safely and avoid the reported error above.
Before performing an offline installation, the necessary packages and files must be created. To do this, just use the -dw
to download the packages and files and the -g
option to generate the certificates:
root@ip-172-31-37-181:/home/ubuntu# bash wazuh-install.sh -dw deb
14/12/2023 14:37:50 INFO: Starting Wazuh installation assistant. Wazuh version: 4.8.0
14/12/2023 14:37:50 INFO: Verbose logging redirected to /var/log/wazuh-install.log
14/12/2023 14:38:11 INFO: Verifying that your system meets the recommended minimum hardware requirements.
14/12/2023 14:38:11 INFO: --- Download Packages ---
14/12/2023 14:38:11 INFO: Starting Wazuh packages download.
14/12/2023 14:38:11 INFO: Downloading Wazuh deb packages for x86_64.
14/12/2023 14:38:17 INFO: The manager package was downloaded.
14/12/2023 14:38:18 INFO: The filebeat package was downloaded.
14/12/2023 14:38:31 INFO: The indexer package was downloaded.
14/12/2023 14:38:34 INFO: The dashboard package was downloaded.
14/12/2023 14:38:34 INFO: The packages are in wazuh-offline/wazuh-packages
14/12/2023 14:38:34 INFO: Downloading configuration files and assets.
14/12/2023 14:38:35 INFO: The resource https://packages-dev.wazuh.com/key/GPG-KEY-WAZUH was downloaded.
14/12/2023 14:38:35 INFO: The resource https://packages-dev.wazuh.com/4.8/tpl/wazuh/filebeat/filebeat.yml was downloaded.
14/12/2023 14:38:36 INFO: The resource https://raw.githubusercontent.com/wazuh/wazuh/4.8.0/extensions/elasticsearch/7.x/wazuh-template.json was downloaded.
14/12/2023 14:38:36 INFO: The resource https://packages-dev.wazuh.com/pre-release/filebeat/wazuh-filebeat-0.3.tar.gz was downloaded.
14/12/2023 14:38:36 INFO: The configuration files and assets are in wazuh-offline.tar.gz
14/12/2023 14:39:35 INFO: You can follow the installation guide here https://documentation.wazuh.com/current/deployment-options/offline-installation.html
root@ip-172-31-37-181:/home/ubuntu# vi config.yml
root@ip-172-31-37-181:/home/ubuntu# bash wazuh-install.sh -g
14/12/2023 14:39:50 INFO: Starting Wazuh installation assistant. Wazuh version: 4.8.0
14/12/2023 14:39:50 INFO: Verbose logging redirected to /var/log/wazuh-install.log
14/12/2023 14:40:01 ERROR: File /home/ubuntu/wazuh-install-files.tar already exists. Please remove it if you want to use a new configuration.
One this two tar files are created, they must be sent to the offline machines.
As the final testing, the offline installation has been tested in 5 instances of EC2 with the internet connection disabled. This has been done creating the TestOffline
SG, which deny all the outbound traffic.
[root@ip-172-31-40-241 ec2-user]# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3074ms
[root@ip-172-31-40-241 ec2-user]#
By petition of the team, in case of offline installation, the import of the Wazuh GPG key and repository have been removed. Two tests in Ubuntu22 and RHEL9 have been performed:
We must consider that, because of the Wazuh Filebeat template fetched via curl
in the WIA, the wazuh-install.sh -s|--start-cluster
option does not work. This will be fixed in https://github.com/wazuh/wazuh-automation/issues/1475, where the Wazuh Filebeat template is not needed to be downloaded.
Currently working on the documentation of these changes in the following branch: https://github.com/wazuh/wazuh-documentation/tree/wp-1422-improve-offline-documentation-4.9
The aim is to create an index for the Offline installation that contains two alternatives: step-by-step installation and using the assistant.
The related PR is: https://github.com/wazuh/wazuh-documentation/pull/6920
While following Installation alternatives - Offline installation may be helpful to provide a couple of scripts to do most of the tasks in an automated way.
The offline installing process can/should be improved by providing customers with a single offline installer script in both the download and offline systems.
Also, the process should improve the transfer of packages from the download system to the offline system by creating a single tar.gz file as output from the download process. As a result, it is easier to copy between offline systems.
The process should include two parts:
related to - #wazuh/12889