Closed rauldpm closed 1 year ago
zbalkan
commented
1 year ago Just a small edge case: Oracle Enterprise Linux is also RHEL based. But the OEL repositories does not have ipa-gothic-fonts package, it needs to be downloaded by other means. Since Wazuh is not supported on OEL at all, packages does not need to handle this issue in the installation script. I am mentioning this for the sake of documentation.
davidcr01
commented
1 year ago First of all, it is needed if all the reported packages exist and are installed correctly in DEB and RPM systems.
yum list installed does not recognize libnss3.so package as installed. :red_circle: There is a problem with the ipa-gothic-fonts package: in CentOS7/RHEL 7 this package is from the official repository. However, in CentOS8/RHEL8 this package is from the Raven repository, so it is not fetched by default when trying to install it.
An issue related to this was opened in the Kibana repository. https://github.com/elastic/kibana/issues/68823. An option could be to install the vlgothic-fonts package instead.
After talking with the team, we decided to ignore this package as it only concerns the Japanese characters, and we can suppose that the Japanese package font is installed in a system that uses Japanese characters.
davidcr01
commented
1 year ago ipa-gothic-fonts package.To test the issue commented above, I tested to install all the packages except ipa-gothic-fonts in a CentOS 8 system, and the report was successfully created and downloaded.
:heavy_check_mark: With this test, it is ensured that ipa-gothic-fonts is not strictly necessary.
davidcr01
commented
1 year ago An error has been found while performing an AIO installation in a fresh CentOS 7 (Vagrant box with a private address added).
24/02/2023 11:33:38 INFO: --- Dependencies ---
24/02/2023 11:33:38 INFO: Installing xorg-x11-fonts-100dpi.
24/02/2023 11:33:42 INFO: Installing xorg-x11-fonts-75dpi.
24/02/2023 11:33:45 INFO: Installing xorg-x11-utils.
24/02/2023 11:33:51 INFO: Installing xorg-x11-fonts-cyrillic.
24/02/2023 11:33:54 INFO: Installing xorg-x11-fonts-Type1.
24/02/2023 11:33:59 INFO: Installing xorg-x11-fonts-misc.
24/02/2023 11:34:06 INFO: Installing fontconfig.
24/02/2023 11:34:06 INFO: Starting Wazuh dashboard installation.
24/02/2023 11:37:13 INFO: Wazuh dashboard installation finished.
24/02/2023 11:37:13 INFO: Wazuh dashboard post-install configuration finished.
24/02/2023 11:37:13 INFO: Starting service wazuh-dashboard.
24/02/2023 11:37:13 INFO: wazuh-dashboard service started.
24/02/2023 11:37:48 INFO: Initializing Wazuh dashboard web application.
24/02/2023 11:37:50 INFO: Wazuh dashboard web application initialized.
24/02/2023 11:37:50 INFO: --- Summary ---
24/02/2023 11:37:50 INFO: You can access the web interface https://192.168.56.115
User: admin
Password: HYicz*va58nk?4llQG2nBRueA3L?k4IS
24/02/2023 11:37:50 INFO: Installation finished.If I navigate to the URL the Wazuh app displays:
{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred."}The journalctl -r wazuh-dashboard.service | grep -i -e "Error" command displays:
:heavy_check_mark: The error was related to the IP of the VM: the 192.168.56.115 did not work, but 192.168.56.114 worked.
:warning: In CentOS 7 I detected that the chromium or chrome package is essential to generate the reports, as suggested in the documentation. Hence, it is necessary to install it in case google-chrome-stable is not installed.
Chromium and Chrome are two alternatives to generate the reports, but in case Chrome is not installed is better to install Chromium as it is lighter.
davidcr01
commented
1 year ago :warning: In a CentOS 7 system, I tested to generate a report installing the dependencies and the chromium-headless package, but the report could not be generated. Instead, I installed the chromium package and the report was generated successfully.
[root@centos8 vagrant]# yum list installed | grep chrom
chromium.x86_64 110.0.5481.77-1.el8 @epel
chromium-common.x86_64 110.0.5481.77-1.el8 @epel
chromium-headless.x86_64 110.0.5481.77-1.el8 @epel This could mean that the headless chrome and the specified dependencies are not the only things necessary to generate reports. A complete installation of Chrome or Chromium is needed.
A new function has been added to the installCommon.sh file:
function installCommon_checkChromium() {
if [ "${sys_type}" == "yum" ]; then
if (! yum list installed 2>/dev/null | grep -q -E ^"google-chrome-stable"\\.) && (yum list installed 2>/dev/null | grep -q -E ^"chromium"\\.); then
dependencies+=(chromium)
fi
elif [ "${sys_type}" == "apt-get" ]; then
if (! apt list --installed 2>/dev/null | grep -q -E ^"google-chrome-stable"\/) && (apt list --installed 2>/dev/null | grep -q -E ^"chromium-browser"\/); then
dependencies+=(chromium-browser)
fi
fi
}This function checks if the Chrome or Chromium browser is installed, and if none of them are installed, it installs Chromium. This function considered the different names of the browsers depending on the OS, notice that:
chromiumchromium-browser.:warning: It seems in DEB-based systems it is not necessary to install Chrome o Chromium. This needs some testing to decide definitely if DEB-based systems need a Chrome alternative.
:red_circle: : Ubuntu 16.04 :green_circle: : Ubuntu 18.04 :green_circle: Ubuntu 20.04 :green_circle: Ubuntu 22.04
With the DEB-based system tests done, I can assure that in these systems neither Chrome nor Chromium are not necessary, but in the test of Ubuntu16 I discovered that:
This shows that Chrome or Chromium are not strictly necessary, but some of their dependencies are. After some research, if I remove the fontconfig package the report generation stops working, so this package is necessary for DEB-based systems.
davidcr01
commented
1 year ago After huge testing and research, headless-chrome is necessary to generate reports. With that, I conclude that it must be a dependency of the Installation Assitant, and have to be installed if Chrome or Chromium are not installed.
I tested the function installCommon_checkChromium and Chromium is installed if none of both tools are not installed:
02/03/2023 12:52:57 INFO: --- Wazuh dashboard ---
02/03/2023 12:53:00 INFO: --- Dependencies ----
02/03/2023 12:53:00 INFO: Installing chromium-browser.
02/03/2023 13:00:36 INFO: Starting Wazuh dashboard installation.
02/03/2023 13:01:36 INFO: Wazuh dashboard installation finished.
02/03/2023 13:01:36 INFO: Wazuh dashboard post-install configuration finished.
02/03/2023 13:01:36 INFO: Starting service wazuh-dashboard.
02/03/2023 13:01:38 INFO: wazuh-dashboard service started.
02/03/2023 13:02:24 INFO: Initializing Wazuh dashboard web application.
02/03/2023 13:02:25 INFO: Wazuh dashboard web application initialized.
02/03/2023 13:02:25 INFO: --- Summary ---
02/03/2023 13:02:25 INFO: You can access the web interface https://<wazuh-dashboard-ip>
User: admin
Password: uiO64FUDfi6S+Z1T9wCmCpOJA*D+Y2Dm
02/03/2023 13:02:25 INFO: Installation finished.
root@vagrant:/home/vagrant# It seems that with this approach, in Ubuntu 22.04 systems the PDF generation report can not be done, even if:
chromium-browser package is installed.
Whereas the rest of the Ubuntu versions that Wazuh supports (16.04, 18.04, 20.04) the PDF generation report can be done.To solve this problem some workarounds are possible:
It is mandatory to test the previous alternative and talk with the team about which is the best option.
davidcr01
commented
1 year ago I have performed the test specified in the previous update and the results are:
:x: Install the Chromium package through repositories failed. I added the bionic repository in the /etc/apt/source.list and the output of the apt-get update command is:
root@vagrant:/home/vagrant# apt-get update
Hit:1 http://us.archive.ubuntu.com/ubuntu jammy InRelease
Get:2 http://us.archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Hit:3 http://us.archive.ubuntu.com/ubuntu jammy-updates InRelease
Err:2 http://us.archive.ubuntu.com/ubuntu bionic InRelease
The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 3B4FE6ACC0B21F32
Hit:4 http://us.archive.ubuntu.com/ubuntu jammy-backports InRelease
Hit:5 http://us.archive.ubuntu.com/ubuntu jammy-security InRelease
Reading package lists... Done
W: GPG error: http://us.archive.ubuntu.com/ubuntu bionic InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 3B4FE6ACC0B21F32
E: The repository 'http://us.archive.ubuntu.com/ubuntu bionic InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
root@vagrant:/home/vagrant# :heavy_check_mark: Install de Chromium package of Ubuntu 18.04 manually succeeded. This alternative works as well as report generation, but installing a package of an older distribution may not be a good idea. The system can upgrade this package to the newer, making the report generation not work again.
:heavy_check_mark: After talking with the team, we decided that the best option is to install Chrome instead of Chromium only in Ubuntu 22.
To perform this action, a new function has been added:
function installCommon_aptInstallChrome() {
chrome_package="/tmp/wazuh-install-files/chrome.deb"
curl -so "${chrome_package}" https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
common_logger "Installing chrome"
installCommon_aptInstall "${chrome_package}"
}The package is downloaded from the official site of google and placed in the temporary directory where Wazuh stores the temporary files, which will be removed at the end of the script.
And the function installCommon_checkChromium has been modified by adding a new conditional:
# Report generation doesn't work with Chromium in Ubuntu 22
if [ "${DIST_NAME}" == "ubuntu" ] && [ "${DIST_VER}" == "22" ]; then
installCommon_aptInstallChrome
else
dependencies=(chromium-browser)
fiWith this change, Chrome will only be installed in Ubuntu 22 and Chromium will be installed in the rest of the systems.
06/03/2023 11:40:21 INFO: Initializing Wazuh dashboard web application.
06/03/2023 11:40:54 INFO: Wazuh dashboard web application initialized.
06/03/2023 11:40:54 INFO: --- Summary ---
06/03/2023 11:40:54 INFO: You can access the web interface https://<wazuh-dashboard-ip>
User: admin
Password: WhyN*BfbG3tirfqs11fLYEYbJ?XM0.fb
06/03/2023 11:40:54 INFO: Installation finished.
root@ubuntu22:/home/vagrant# apt list --installed | grep chro
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
google-chrome-stable/stable,now 110.0.5481.177-1 amd64 [installed]
root@ubuntu22:/home/vagrant# chromium package.
sudo amazon-linux-extras install epel -y
This command is added to the function installCommon_checkChromiumWith this, report generation in AL2 works as expected.
07/03/2023 09:10:13 INFO: --- Wazuh dashboard ---
07/03/2023 09:10:21 INFO: --- Dependencies ---
07/03/2023 09:10:21 INFO: Installing chromium.
07/03/2023 09:10:47 INFO: Installing xorg-x11-fonts-100dpi.
07/03/2023 09:10:54 INFO: Installing xorg-x11-fonts-75dpi.
07/03/2023 09:10:59 INFO: Installing xorg-x11-utils.
07/03/2023 09:11:04 INFO: Installing xorg-x11-fonts-cyrillic.
07/03/2023 09:11:10 INFO: Installing xorg-x11-fonts-Type1.
07/03/2023 09:11:15 INFO: Installing xorg-x11-fonts-misc.
07/03/2023 09:11:23 INFO: Starting Wazuh dashboard installation.
07/03/2023 09:12:52 INFO: Wazuh dashboard installation finished.
07/03/2023 09:12:52 INFO: Wazuh dashboard post-install configuration finished.
07/03/2023 09:12:52 INFO: Starting service wazuh-dashboard.
07/03/2023 09:12:52 INFO: wazuh-dashboard service started.
07/03/2023 09:13:21 INFO: Initializing Wazuh dashboard web application.
07/03/2023 09:13:22 INFO: Wazuh dashboard web application initialized.
07/03/2023 09:13:22 INFO: --- Summary ---
07/03/2023 09:13:22 INFO: You can access the web interface https://<wazuh-dashboard-ip>
User: admin
Password: 5mlT3gHQkgrkOKY16NLpAVc.SNnSzPOG
07/03/2023 09:13:22 INFO: Installation finished.:heavy_check_mark: : With the correct installation, the generation of reports works:
:x: In Ubuntu 20.04, the PDF generation report does not work installing Chromium but it works with Chrome (similar to the behavior of Ubuntu 22).
Hence, Ubuntu 20 will be taken into count in the unattended script to install Chrome instead of Chromium.
davidcr01
commented
1 year ago To test the installation in these systems, I must add the CentOS repositories to the list of available repositories. I think these notes could be useful if we need to install extra dependencies that are not in the default RHEL repositories or in the EPEL repository.
RHEL 8
[extras]
name=CentOS Linux $releasever - Extras
baseurl=http://vault.centos.org/centos/$releasever/extras/$basearch/os/
gpgcheck=1
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
[baseos]
name=CentOS Linux 8 - BaseOS
baseurl=http://vault.centos.org/centos/8/BaseOS/$basearch/os/
gpgcheck=1
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
[appstream]
name=CentOS Linux 8 - AppStream
baseurl=http://vault.centos.org/centos/8/AppStream/$basearch/os/
gpgcheck=1
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficialRHEL 9
[appstream]
name=CentOS Stream $releasever - AppStream
baseurl=https://mirror.stream.centos.org/9-stream/AppStream/x86_64/os/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
gpgcheck=1
repo_gpgcheck=0
metadata_expire=6h
countme=1
enabled=1
[baseos]
name=CentOS Stream $releasever - BaseOS
baseurl=https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/os/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
gpgcheck=1
repo_gpgcheck=0
metadata_expire=6h
countme=1
enabled=1To summarize, the workaround that has been done is:
ipa-gothic-fonts from the necessary packages as it is not needed. It only concerns the Japanese characters and we can suppose that a Japanese system will have a Japanese characters package.installCommon_checkChromium and installCommon_aptInstallChrome.Besides, I created a simple script that adds the Centos repositories to the RHEL8 and RHEL9 systems:
davidcr01
commented
1 year ago After the requested changes of @DFolchA I have created a new logic to allow optional installations. This is done by some flags that indicate this type of installation. The use of flags is to avoid creating new functions that may repeat code.
Every time the installation is going to be optional, the flag optional_installation must be set to 1, and set to 0 when that optional installation is completed.
In this case, the installation of the PDF report dependencies is optional as they are not essential to make the Wazuh dashboard work. To warn this, if an optional dependency installation fails, the flag report_dependencies is set to 1 to make the Installation Assistant print a warning message.
This has been done in https://github.com/wazuh/wazuh-packages/pull/2122/commits/053e1387a6ce55dd2bc5ad8ff4d9d6e30ecd0980.
To test this behavior, and force the failure of an optional installation dependency, I have tested in Ubuntu 20.04 with the Chrome package:
Notice that the Chrome installation has failed and notified with WARNING: Cannot install optional dependency: chrome., and after, another warning is displayed: WARNING: Dependencies skipped. PDF report generation may not work.
zbalkan
commented
1 year ago As an external person or a user, I would like to point that adding new repositories is a system-wide change, which must be explicit as possible: both during the installation and within documentation. Because there will be conflicts. Also in corporate use cases where a local repository is used, the dependencies should be explicit so the admins should add them to their local repository.
davidcr01
commented
1 year ago Hello @zbalkan.
If you refer to the Google repository to install Chrome, this repository has not been added by the consequences you mention. To avoid this, the Chrome package is downloaded and installed from the official repository.
If you refer to the EPEL package in Amazon Linux 2, this has been changed as it is not a good idea to add new repositories. The solution for this is what I commented before, to install the tool by downloading the package. This has been done in https://github.com/wazuh/wazuh-packages/pull/2122/commits/2095105006b6f3e1965c387a4b89756cbf561dde
If you are interested in the changes made in this issue, you can read the summary comment https://github.com/wazuh/wazuh-packages/issues/1840#issuecomment-1461569918 or the changed files of the associated PR: https://github.com/wazuh/wazuh-packages/pull/2122/files
Hope it helps. Regards. The Wazuh team.
davidcr01
commented
1 year ago Due to the recommendation of @zbalkan and after talking with the team, we decided to remove the amazon-linux-extras install epel -y command and install Chrome by downloading it from the official site, as done in some Ubuntu systems.
:green_circle: The installation and generation of PDF reports are correct in Amazon Linux 2:
zbalkan
commented
1 year ago Dear @davidcr01 ,
Thank you for considering my feedback. I would like to ask about one thing, when installation goes south. If the server is isolated from internet and downloads over curl fail, what would be the action? According to the PR, it looks like it will just exit with a warning. Yet, IMHO, this is not a big issue to break the installation. With a proper and clear warning, preferably after a prompt to continue, a user should be able to continue installation without the missing package, knowing that they have to solve the issue over their local repository.
davidcr01
commented
1 year ago Hello @zbalkan
The behavior of the Installation Assistant, in that case, would be the following:
curl command to our common_curl function. This function retries the specified curl several times in case a transient error is generated. If there is an internet connection loss, it will retry the curl X times until a timeout is reached, and then the warning message would be displayed. If the timeout is not reached yet and there is internet connection, the package would be successfully downloaded.I hope I clarified your question. Regards. David.
Main issue: https://github.com/wazuh/wazuh-packages/issues/1520
It is necessary to include in the Wazuh dashboard installation process the installation of the necessary dependencies to generate PDF reports, which according to the OpenSearch documentation are:
In Red Hat systems it is necessary to register the system through subscription, if it is not done the packages will not be found when installed, in this https://github.com/wazuh/wazuh-packages/issues/1520#issuecomment-1240928321 it is verified that through subscription the installation is carried out correctly, so it would be interesting to add a note about these systems.
It is also necessary to decide whether or not the installation should continue in case of finding an error in these recommended systems when installing the dependencies.
Related links:
Resolution
The summary of the resolution of this issue is in: https://github.com/wazuh/wazuh-packages/issues/1840#issuecomment-1461569918
Testing
The testing for the resolution of this issue is in: https://github.com/wazuh/wazuh-packages/issues/1840#issuecomment-1441654661