search

wazuh / wazuh-packages

Wazuh - Tools for packages creation
https://wazuh.com
GNU General Public License v2.0
105 stars 95 forks source link

Improve package revision in Offline download method #3109

Closed CarlosALgit closed 1 month ago

CarlosALgit commented 1 month ago
Wazuh version Install type Action performed Platform
4.9.0 Manager Install rpm/deb

While working in this issue https://github.com/wazuh/wazuh-packages/issues/3093 I found that the Offline download method fails because it has the revision of the Wazuh central components hardcoded. So, as for 4.9.0 we have the Wazuh Dashboard package with revision 2, it doesn't find it because it's looking for revision 1 and the download of the packages fails.

[!NOTE] It happens on both rpm and deb as can be seen on this logs:

➜  packages-3093 sudo ./wazuh-install.sh -dw deb
10/09/2024 10:39:24 INFO: Starting Wazuh installation assistant. Wazuh version: 4.9.0
10/09/2024 10:39:24 INFO: Verbose logging redirected to /var/log/wazuh-install.log
10/09/2024 10:39:29 INFO: Verifying that your system meets the recommended minimum hardware requirements.
10/09/2024 10:39:29 INFO: --- Download Packages ---
10/09/2024 10:39:29 INFO: Starting Wazuh packages download.
10/09/2024 10:39:29 INFO: Downloading Wazuh deb packages for x86_64.
10/09/2024 10:39:36 INFO: The manager package was downloaded.
10/09/2024 10:39:37 INFO: The filebeat package was downloaded.
10/09/2024 10:39:49 INFO: The indexer package was downloaded.
10/09/2024 10:39:50 ERROR: The dashboard package could not be downloaded. Exiting.
➜  rpm sudo ./wazuh-install.sh -dw rpm
10/09/2024 10:52:53 INFO: Starting Wazuh installation assistant. Wazuh version: 4.9.0
10/09/2024 10:52:53 INFO: Verbose logging redirected to /var/log/wazuh-install.log
10/09/2024 10:53:00 INFO: Verifying that your system meets the recommended minimum hardware requirements.
10/09/2024 10:53:00 INFO: --- Download Packages ---
10/09/2024 10:53:00 INFO: Starting Wazuh packages download.
10/09/2024 10:53:00 INFO: Downloading Wazuh rpm packages for x86_64.
10/09/2024 10:53:07 INFO: The manager package was downloaded.
10/09/2024 10:53:08 INFO: The filebeat package was downloaded.
10/09/2024 10:53:20 INFO: The indexer package was downloaded.
10/09/2024 10:53:21 ERROR: The dashboard package could not be downloaded. Exiting.

The fix needed is to change the revision and look for the latest revision of each package as done here: https://github.com/wazuh/wazuh-puppet/issues/1105.

c-bordon commented 1 month ago

I was able to replicate it, the problem occurs because the package with revision 1 does not exist in the production repository, so the validation as such cannot be performed. We will have to validate if it is correct that package revision 1 does not exist, if this is so, we will have to modify the logic to validate the existence of different revisions.

rauldpm commented 1 month ago

We will have to validate if it is correct that package revision 1 does not exist,

Yes, the package should not exist and it is the reason we released a -2 Wazuh dashboard package

c-bordon commented 1 month ago

Update report

Considering that the current validation method is through curl, the current validation is useless since the production repository only keeps the latest package, that is, the latest revision.

One of the options to validate the available package is through yum and apt, although this can have some complications:

options with yum:

[root@rhel8 ~]# yum list available wazuh-indexer-4.9.0
Last metadata expiration check: 0:34:25 ago on Tue 10 Sep 2024 07:14:13 PM UTC.
Available Packages
wazuh-indexer.x86_64                                                                                                      4.9.0-1                                                                                                      wazuh
[root@rhel8 ~]# yum list available wazuh-dashboard-4.9.0
Last metadata expiration check: 0:34:28 ago on Tue 10 Sep 2024 07:14:13 PM UTC.
Available Packages
wazuh-dashboard.x86_64                                                                                                     4.9.0-2                                                                                                     wazuh
[root@rhel8 ~]# yum list available wazuh-manager-4.9.0
Last metadata expiration check: 0:34:31 ago on Tue 10 Sep 2024 07:14:13 PM UTC.
Available Packages
wazuh-manager.x86_64                                                                                                      4.9.0-1                                                                                                      wazuh

options with apt:

root@ubuntu-jammy:~# apt-cache madison wazuh-dashboard
wazuh-dashboard |    4.9.0-2 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.8.2-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.8.1-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.8.0-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.7.5-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.7.4-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.7.3-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.7.2-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.7.1-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.7.0-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.6.0-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.5.4-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.5.3-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.5.2-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.5.1-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.5.0-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.4.5-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.4.4-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.4.3-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.4.2-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.4.1-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.4.0-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |   4.3.11-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |   4.3.10-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.3.9-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.3.8-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.3.7-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.3.6-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.3.5-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.3.4-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.3.3-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.3.2-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.3.1-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
wazuh-dashboard |    4.3.0-1 | https://packages.wazuh.com/4.x/apt stable/main amd64 Packages
root@ubuntu-jammy:~# apt search wazuh-dashboard
Sorting... Done
Full Text Search... Done
wazuh-dashboard/stable 4.9.0-2 amd64
  Wazuh dashboard is a user interface and visualization tool for security-related data. This Wazuh central component enables exploring, visualizing, and analyzing the stored security alerts generated by the Wazuh server. Wazuh dashboard enables inspecting the status and managing the configurations of the Wazuh cluster and agents as well as creating and managing users and roles. In addition, it allows testing the ruleset and making calls to the Wazuh API. Documentation can be found at https://documentation.wazuh.com/current/getting-started/components/wazuh-dashboard.html
root@ubuntu-jammy:~# apt list wazuh-manager
Listing... Done
wazuh-manager/stable 4.9.0-1 amd64
N: There are 52 additional versions. Please use the '-a' switch to see them.
root@ubuntu-jammy:~# apt list wazuh-dashboard
Listing... Done
wazuh-dashboard/stable 4.9.0-2 amd64
N: There are 33 additional versions. Please use the '-a' switch to see them.
root@ubuntu-jammy:~# apt list wazuh-indexer
Listing... Done
wazuh-indexer/stable 4.9.0-1 amd64
N: There are 33 additional versions. Please use the '-a' switch to see them.

with apt we cannot pass the specific version:

N: There are 33 additional versions. Please use the '-a' switch to see them.
root@ubuntu-jammy:~# apt list wazuh-indexer_4.9.0
Listing... Error!
E: input:0-19: error: Expected pattern
   wazuh-indexer_4.9.0
   ^^^^^^^^^^^^^^^^^^^
root@ubuntu-jammy:~# apt list wazuh-indexer=4.9.0
Listing... Error!
E: input:0-19: error: Expected pattern
   wazuh-indexer=4.9.0
   ^^^^^^^^^^^^^^^^^^^
root@ubuntu-jammy:~# apt list wazuh-indexer-4.9.0
Listing... Done
c-bordon commented 1 month ago

Update report

The fix for this case is quite complex due to various situations present in the publishing process.

The main problem is that when trying to download the package with cURL, we cannot use wildcards or obtain the list of files available in the bucket, therefore, we cannot obtain what the latest revision is.

The current approach performs an incremental validation by packages and revision, starting with the package with revision 1, if it finds it, it performs the validation of the existence of a package with revision 2, if it does not find it, it keeps the package with revision 1 as the last available. This approach breaks with the publishing process where in packages.wazuh.com only the latest package is available (and not all its previous revisions), hence the error detected in this issue.

Validating the available packages with the APT or YUM package manager is also not valid, since the user who is downloading the packages may have a different distribution than the one on the machines where Wazuh is going to be installed, so this solution is not valid either.

The possible alternatives are:

1- Remove the current validation process and hardcode the revision of each package, this implies that every time we have to release a package with a revision other than 1, we must also update the wizard and also release the wizard (similar to what happened in the 4.9.0 release) 2- Establish an X ​​number of tolerable revisions (CTO can define this number), where it is understood that if a package requires more than 10 revisions, we must directly release a patch for the minor version and not a package revision. 3- Use some dependency other than cURL, such as wget or AWS CLI to obtain the list of packages, but this is something we do not want to do with the installation assistant 4- Keep the same current logic for pre-release and create a new logic for production, this logic should get the list of published packages from this documentation URL https://documentation.wazuh.com/current/installation-guide/packages-list.html, process the result to get the published packages. The downside is that we depend on the availability of the documentation and that it does not suffer major changes to match with different parameters needed to get the package.