search

wazuh / wazuh-docker

Wazuh - Docker containers
https://wazuh.com
Other
694 stars 397 forks source link

Release Wazuh Indexer into Docker deploy #567

Closed vcerenu closed 2 years ago

vcerenu commented 2 years ago

Analysis and creation of image for deploy Wazuh indexer into Docker deploy (production-cluster.yml)

vcerenu commented 2 years ago

Branch https://github.com/wazuh/wazuh-docker/tree/new-packages-release was created where the necessary files are generated to build the Docker image for wazuh-indexer.

The Dockerfile was created for it and the necessary files for the configuration of the Docker image are being created, in addition to the correct entrypoint.

├── wazuh-indexer
│   ├── config
│   │   ├── entrypoint_odfe.sh
│   │   ├── entrypoint_prueba.sh
│   │   ├── entrypoint.sh
│   │   └── wazuh.repo
│   ├── Dockerfile
│   └── Dockerfile_ubuntu
vcerenu commented 2 years ago

The Dockerfile was created based on Focal Ubuntu. When installing the wazuh-indexer package it throws the following error:

$ docker build -t wazuh/wazuh-indexer .
Sending build context to Docker daemon  13.82kB
Step 1/12 : FROM ubuntu:20.04
 ---> d13c942271d6
Step 2/12 : ARG WAZUH_VERSION=4.3.0
 ---> Running in 3cd662f1d6d9
Removing intermediate container 3cd662f1d6d9
 ---> 62a7bb70c1c1
Step 3/12 : ARG TEMPLATE_VERSION="master"
 ---> Running in 69f8a5ce7162
Removing intermediate container 69f8a5ce7162
 ---> 0a21f220e254
Step 4/12 : ARG FILEBEAT_CHANNEL=filebeat-oss
 ---> Running in 3e0dad85ec0e
Removing intermediate container 3e0dad85ec0e
 ---> fa847d680b08
Step 5/12 : ARG FILEBEAT_VERSION=7.10.2
 ---> Running in 1fe134567037
Removing intermediate container 1fe134567037
 ---> 417d9330061f
Step 6/12 : ARG WAZUH_FILEBEAT_MODULE="wazuh-filebeat-0.1.tar.gz"
 ---> Running in 6cf3b538933a
Removing intermediate container 6cf3b538933a
 ---> 6b19f640fc28
Step 7/12 : RUN apt-get update && apt install curl gnupg -y
 ---> Running in e5864b0382ee
Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Get:2 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [30.1 kB]
Get:3 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [889 kB]
Get:4 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [837 kB]
Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1468 kB]
Get:6 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:7 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:8 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB]
Get:9 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:10 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:13 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [952 kB]
Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1119 kB]
Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1894 kB]
Get:16 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [33.7 kB]
Get:17 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [22.4 kB]
Get:18 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [50.8 kB]
Fetched 20.7 MB in 14s (1449 kB/s)
Reading package lists...

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  ca-certificates dirmngr gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client
  gpg-wks-server gpgconf gpgsm krb5-locales libasn1-8-heimdal libassuan0
  libbrotli1 libcurl4 libgssapi-krb5-2 libgssapi3-heimdal libhcrypto4-heimdal
  libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libk5crypto3
  libkeyutils1 libkrb5-26-heimdal libkrb5-3 libkrb5support0 libksba8
  libldap-2.4-2 libldap-common libnghttp2-14 libnpth0 libpsl5 libreadline8
  libroken18-heimdal librtmp1 libsasl2-2 libsasl2-modules libsasl2-modules-db
  libsqlite3-0 libssh-4 libssl1.1 libwind0-heimdal openssl pinentry-curses
  publicsuffix readline-common
Suggested packages:
  dbus-user-session libpam-systemd pinentry-gnome3 tor parcimonie xloadimage
  scdaemon krb5-doc krb5-user libsasl2-modules-gssapi-mit
  | libsasl2-modules-gssapi-heimdal libsasl2-modules-ldap libsasl2-modules-otp
  libsasl2-modules-sql pinentry-doc readline-doc
The following NEW packages will be installed:
  ca-certificates curl dirmngr gnupg gnupg-l10n gnupg-utils gpg gpg-agent
  gpg-wks-client gpg-wks-server gpgconf gpgsm krb5-locales libasn1-8-heimdal
  libassuan0 libbrotli1 libcurl4 libgssapi-krb5-2 libgssapi3-heimdal
  libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal
  libhx509-5-heimdal libk5crypto3 libkeyutils1 libkrb5-26-heimdal libkrb5-3
  libkrb5support0 libksba8 libldap-2.4-2 libldap-common libnghttp2-14 libnpth0
  libpsl5 libreadline8 libroken18-heimdal librtmp1 libsasl2-2 libsasl2-modules
  libsasl2-modules-db libsqlite3-0 libssh-4 libssl1.1 libwind0-heimdal openssl
  pinentry-curses publicsuffix readline-common
0 upgraded, 48 newly installed, 0 to remove and 2 not upgraded.
Need to get 8168 kB of archives.
After this operation, 24.7 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.10 [1322 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 openssl amd64 1.1.1f-1ubuntu2.10 [620 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ca-certificates all 20210119~20.04.2 [145 kB]
Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 readline-common all 8.0-4 [53.5 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 libreadline8 amd64 8.0-4 [131 kB]
Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libsqlite3-0 amd64 3.31.1-4ubuntu0.2 [549 kB]
Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 krb5-locales all 1.17-6ubuntu4.1 [11.4 kB]
Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libkrb5support0 amd64 1.17-6ubuntu4.1 [30.9 kB]
Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libk5crypto3 amd64 1.17-6ubuntu4.1 [79.9 kB]
Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libkeyutils1 amd64 1.6-6ubuntu1 [10.2 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libkrb5-3 amd64 1.17-6ubuntu4.1 [330 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libgssapi-krb5-2 amd64 1.17-6ubuntu4.1 [121 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 libpsl5 amd64 0.21.0-1ubuntu1 [51.5 kB]
Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 publicsuffix all 20200303.0012-1 [111 kB]
Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbrotli1 amd64 1.0.7-6ubuntu0.1 [267 kB]
Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 libroken18-heimdal amd64 7.7.0+dfsg-1ubuntu1 [41.8 kB]
Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 libasn1-8-heimdal amd64 7.7.0+dfsg-1ubuntu1 [181 kB]
Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 libheimbase1-heimdal amd64 7.7.0+dfsg-1ubuntu1 [29.7 kB]
Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 libhcrypto4-heimdal amd64 7.7.0+dfsg-1ubuntu1 [87.9 kB]
Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 libwind0-heimdal amd64 7.7.0+dfsg-1ubuntu1 [48.0 kB]
Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libhx509-5-heimdal amd64 7.7.0+dfsg-1ubuntu1 [107 kB]
Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libkrb5-26-heimdal amd64 7.7.0+dfsg-1ubuntu1 [208 kB]
Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 libheimntlm0-heimdal amd64 7.7.0+dfsg-1ubuntu1 [15.1 kB]
Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 libgssapi3-heimdal amd64 7.7.0+dfsg-1ubuntu1 [96.1 kB]
Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 libsasl2-modules-db amd64 2.1.27+dfsg-2 [14.9 kB]
Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 libsasl2-2 amd64 2.1.27+dfsg-2 [49.3 kB]
Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libldap-common all 2.4.49+dfsg-2ubuntu1.8 [16.6 kB]
Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libldap-2.4-2 amd64 2.4.49+dfsg-2ubuntu1.8 [155 kB]
Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 libnghttp2-14 amd64 1.40.0-1build1 [78.7 kB]
Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 librtmp1 amd64 2.4+20151223.gitfa8646d.1-2build1 [54.9 kB]
Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssh-4 amd64 0.9.3-2ubuntu2.2 [170 kB]
Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libcurl4 amd64 7.68.0-1ubuntu2.7 [234 kB]
Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 curl amd64 7.68.0-1ubuntu2.7 [161 kB]
Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 libassuan0 amd64 2.5.3-7ubuntu2 [35.7 kB]
Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gpgconf amd64 2.2.19-3ubuntu2.1 [124 kB]
Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 libksba8 amd64 1.3.5-2 [92.6 kB]
Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 libnpth0 amd64 1.6-1 [7736 B]
Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 dirmngr amd64 2.2.19-3ubuntu2.1 [329 kB]
Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gnupg-l10n all 2.2.19-3ubuntu2.1 [51.7 kB]
Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gnupg-utils amd64 2.2.19-3ubuntu2.1 [480 kB]
Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gpg amd64 2.2.19-3ubuntu2.1 [483 kB]
Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 pinentry-curses amd64 1.1.0-3build1 [36.3 kB]
Get:43 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gpg-agent amd64 2.2.19-3ubuntu2.1 [232 kB]
Get:44 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gpg-wks-client amd64 2.2.19-3ubuntu2.1 [97.6 kB]
Get:45 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gpg-wks-server amd64 2.2.19-3ubuntu2.1 [90.3 kB]
Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gpgsm amd64 2.2.19-3ubuntu2.1 [217 kB]
Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 gnupg all 2.2.19-3ubuntu2.1 [259 kB]
Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 libsasl2-modules amd64 2.1.27+dfsg-2 [49.1 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 8168 kB in 6s (1264 kB/s)
Selecting previously unselected package libssl1.1:amd64.
(Reading database ... 4127 files and directories currently installed.)
Preparing to unpack .../00-libssl1.1_1.1.1f-1ubuntu2.10_amd64.deb ...
Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.10) ...
Selecting previously unselected package openssl.
Preparing to unpack .../01-openssl_1.1.1f-1ubuntu2.10_amd64.deb ...
Unpacking openssl (1.1.1f-1ubuntu2.10) ...
Selecting previously unselected package ca-certificates.
Preparing to unpack .../02-ca-certificates_20210119~20.04.2_all.deb ...
Unpacking ca-certificates (20210119~20.04.2) ...
Selecting previously unselected package readline-common.
Preparing to unpack .../03-readline-common_8.0-4_all.deb ...
Unpacking readline-common (8.0-4) ...
Selecting previously unselected package libreadline8:amd64.
Preparing to unpack .../04-libreadline8_8.0-4_amd64.deb ...
Unpacking libreadline8:amd64 (8.0-4) ...
Selecting previously unselected package libsqlite3-0:amd64.
Preparing to unpack .../05-libsqlite3-0_3.31.1-4ubuntu0.2_amd64.deb ...
Unpacking libsqlite3-0:amd64 (3.31.1-4ubuntu0.2) ...
Selecting previously unselected package krb5-locales.
Preparing to unpack .../06-krb5-locales_1.17-6ubuntu4.1_all.deb ...
Unpacking krb5-locales (1.17-6ubuntu4.1) ...
Selecting previously unselected package libkrb5support0:amd64.
Preparing to unpack .../07-libkrb5support0_1.17-6ubuntu4.1_amd64.deb ...
Unpacking libkrb5support0:amd64 (1.17-6ubuntu4.1) ...
Selecting previously unselected package libk5crypto3:amd64.
Preparing to unpack .../08-libk5crypto3_1.17-6ubuntu4.1_amd64.deb ...
Unpacking libk5crypto3:amd64 (1.17-6ubuntu4.1) ...
Selecting previously unselected package libkeyutils1:amd64.
Preparing to unpack .../09-libkeyutils1_1.6-6ubuntu1_amd64.deb ...
Unpacking libkeyutils1:amd64 (1.6-6ubuntu1) ...
Selecting previously unselected package libkrb5-3:amd64.
Preparing to unpack .../10-libkrb5-3_1.17-6ubuntu4.1_amd64.deb ...
Unpacking libkrb5-3:amd64 (1.17-6ubuntu4.1) ...
Selecting previously unselected package libgssapi-krb5-2:amd64.
Preparing to unpack .../11-libgssapi-krb5-2_1.17-6ubuntu4.1_amd64.deb ...
Unpacking libgssapi-krb5-2:amd64 (1.17-6ubuntu4.1) ...
Selecting previously unselected package libpsl5:amd64.
Preparing to unpack .../12-libpsl5_0.21.0-1ubuntu1_amd64.deb ...
Unpacking libpsl5:amd64 (0.21.0-1ubuntu1) ...
Selecting previously unselected package publicsuffix.
Preparing to unpack .../13-publicsuffix_20200303.0012-1_all.deb ...
Unpacking publicsuffix (20200303.0012-1) ...
Selecting previously unselected package libbrotli1:amd64.
Preparing to unpack .../14-libbrotli1_1.0.7-6ubuntu0.1_amd64.deb ...
Unpacking libbrotli1:amd64 (1.0.7-6ubuntu0.1) ...
Selecting previously unselected package libroken18-heimdal:amd64.
Preparing to unpack .../15-libroken18-heimdal_7.7.0+dfsg-1ubuntu1_amd64.deb ...
Unpacking libroken18-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Selecting previously unselected package libasn1-8-heimdal:amd64.
Preparing to unpack .../16-libasn1-8-heimdal_7.7.0+dfsg-1ubuntu1_amd64.deb ...
Unpacking libasn1-8-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Selecting previously unselected package libheimbase1-heimdal:amd64.
Preparing to unpack .../17-libheimbase1-heimdal_7.7.0+dfsg-1ubuntu1_amd64.deb ...
Unpacking libheimbase1-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Selecting previously unselected package libhcrypto4-heimdal:amd64.
Preparing to unpack .../18-libhcrypto4-heimdal_7.7.0+dfsg-1ubuntu1_amd64.deb ...
Unpacking libhcrypto4-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Selecting previously unselected package libwind0-heimdal:amd64.
Preparing to unpack .../19-libwind0-heimdal_7.7.0+dfsg-1ubuntu1_amd64.deb ...
Unpacking libwind0-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Selecting previously unselected package libhx509-5-heimdal:amd64.
Preparing to unpack .../20-libhx509-5-heimdal_7.7.0+dfsg-1ubuntu1_amd64.deb ...
Unpacking libhx509-5-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Selecting previously unselected package libkrb5-26-heimdal:amd64.
Preparing to unpack .../21-libkrb5-26-heimdal_7.7.0+dfsg-1ubuntu1_amd64.deb ...
Unpacking libkrb5-26-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Selecting previously unselected package libheimntlm0-heimdal:amd64.
Preparing to unpack .../22-libheimntlm0-heimdal_7.7.0+dfsg-1ubuntu1_amd64.deb ...
Unpacking libheimntlm0-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Selecting previously unselected package libgssapi3-heimdal:amd64.
Preparing to unpack .../23-libgssapi3-heimdal_7.7.0+dfsg-1ubuntu1_amd64.deb ...
Unpacking libgssapi3-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Selecting previously unselected package libsasl2-modules-db:amd64.
Preparing to unpack .../24-libsasl2-modules-db_2.1.27+dfsg-2_amd64.deb ...
Unpacking libsasl2-modules-db:amd64 (2.1.27+dfsg-2) ...
Selecting previously unselected package libsasl2-2:amd64.
Preparing to unpack .../25-libsasl2-2_2.1.27+dfsg-2_amd64.deb ...
Unpacking libsasl2-2:amd64 (2.1.27+dfsg-2) ...
Selecting previously unselected package libldap-common.
Preparing to unpack .../26-libldap-common_2.4.49+dfsg-2ubuntu1.8_all.deb ...
Unpacking libldap-common (2.4.49+dfsg-2ubuntu1.8) ...
Selecting previously unselected package libldap-2.4-2:amd64.
Preparing to unpack .../27-libldap-2.4-2_2.4.49+dfsg-2ubuntu1.8_amd64.deb ...
Unpacking libldap-2.4-2:amd64 (2.4.49+dfsg-2ubuntu1.8) ...
Selecting previously unselected package libnghttp2-14:amd64.
Preparing to unpack .../28-libnghttp2-14_1.40.0-1build1_amd64.deb ...
Unpacking libnghttp2-14:amd64 (1.40.0-1build1) ...
Selecting previously unselected package librtmp1:amd64.
Preparing to unpack .../29-librtmp1_2.4+20151223.gitfa8646d.1-2build1_amd64.deb ...
Unpacking librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2build1) ...
Selecting previously unselected package libssh-4:amd64.
Preparing to unpack .../30-libssh-4_0.9.3-2ubuntu2.2_amd64.deb ...
Unpacking libssh-4:amd64 (0.9.3-2ubuntu2.2) ...
Selecting previously unselected package libcurl4:amd64.
Preparing to unpack .../31-libcurl4_7.68.0-1ubuntu2.7_amd64.deb ...
Unpacking libcurl4:amd64 (7.68.0-1ubuntu2.7) ...
Selecting previously unselected package curl.
Preparing to unpack .../32-curl_7.68.0-1ubuntu2.7_amd64.deb ...
Unpacking curl (7.68.0-1ubuntu2.7) ...
Selecting previously unselected package libassuan0:amd64.
Preparing to unpack .../33-libassuan0_2.5.3-7ubuntu2_amd64.deb ...
Unpacking libassuan0:amd64 (2.5.3-7ubuntu2) ...
Selecting previously unselected package gpgconf.
Preparing to unpack .../34-gpgconf_2.2.19-3ubuntu2.1_amd64.deb ...
Unpacking gpgconf (2.2.19-3ubuntu2.1) ...
Selecting previously unselected package libksba8:amd64.
Preparing to unpack .../35-libksba8_1.3.5-2_amd64.deb ...
Unpacking libksba8:amd64 (1.3.5-2) ...
Selecting previously unselected package libnpth0:amd64.
Preparing to unpack .../36-libnpth0_1.6-1_amd64.deb ...
Unpacking libnpth0:amd64 (1.6-1) ...
Selecting previously unselected package dirmngr.
Preparing to unpack .../37-dirmngr_2.2.19-3ubuntu2.1_amd64.deb ...
Unpacking dirmngr (2.2.19-3ubuntu2.1) ...
Selecting previously unselected package gnupg-l10n.
Preparing to unpack .../38-gnupg-l10n_2.2.19-3ubuntu2.1_all.deb ...
Unpacking gnupg-l10n (2.2.19-3ubuntu2.1) ...
Selecting previously unselected package gnupg-utils.
Preparing to unpack .../39-gnupg-utils_2.2.19-3ubuntu2.1_amd64.deb ...
Unpacking gnupg-utils (2.2.19-3ubuntu2.1) ...
Selecting previously unselected package gpg.
Preparing to unpack .../40-gpg_2.2.19-3ubuntu2.1_amd64.deb ...
Unpacking gpg (2.2.19-3ubuntu2.1) ...
Selecting previously unselected package pinentry-curses.
Preparing to unpack .../41-pinentry-curses_1.1.0-3build1_amd64.deb ...
Unpacking pinentry-curses (1.1.0-3build1) ...
Selecting previously unselected package gpg-agent.
Preparing to unpack .../42-gpg-agent_2.2.19-3ubuntu2.1_amd64.deb ...
Unpacking gpg-agent (2.2.19-3ubuntu2.1) ...
Selecting previously unselected package gpg-wks-client.
Preparing to unpack .../43-gpg-wks-client_2.2.19-3ubuntu2.1_amd64.deb ...
Unpacking gpg-wks-client (2.2.19-3ubuntu2.1) ...
Selecting previously unselected package gpg-wks-server.
Preparing to unpack .../44-gpg-wks-server_2.2.19-3ubuntu2.1_amd64.deb ...
Unpacking gpg-wks-server (2.2.19-3ubuntu2.1) ...
Selecting previously unselected package gpgsm.
Preparing to unpack .../45-gpgsm_2.2.19-3ubuntu2.1_amd64.deb ...
Unpacking gpgsm (2.2.19-3ubuntu2.1) ...
Selecting previously unselected package gnupg.
Preparing to unpack .../46-gnupg_2.2.19-3ubuntu2.1_all.deb ...
Unpacking gnupg (2.2.19-3ubuntu2.1) ...
Selecting previously unselected package libsasl2-modules:amd64.
Preparing to unpack .../47-libsasl2-modules_2.1.27+dfsg-2_amd64.deb ...
Unpacking libsasl2-modules:amd64 (2.1.27+dfsg-2) ...
Setting up libksba8:amd64 (1.3.5-2) ...
Setting up libkeyutils1:amd64 (1.6-6ubuntu1) ...
Setting up libpsl5:amd64 (0.21.0-1ubuntu1) ...
Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.10) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.30.0 /usr/local/share/perl/5.30.0 /usr/lib/x86_64-linux-gnu/perl5/5.30 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.30 /usr/share/perl/5.30 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)
debconf: falling back to frontend: Teletype
Setting up libbrotli1:amd64 (1.0.7-6ubuntu0.1) ...
Setting up libsqlite3-0:amd64 (3.31.1-4ubuntu0.2) ...
Setting up libsasl2-modules:amd64 (2.1.27+dfsg-2) ...
Setting up libnghttp2-14:amd64 (1.40.0-1build1) ...
Setting up libnpth0:amd64 (1.6-1) ...
Setting up krb5-locales (1.17-6ubuntu4.1) ...
Setting up libassuan0:amd64 (2.5.3-7ubuntu2) ...
Setting up libldap-common (2.4.49+dfsg-2ubuntu1.8) ...
Setting up libkrb5support0:amd64 (1.17-6ubuntu4.1) ...
Setting up libsasl2-modules-db:amd64 (2.1.27+dfsg-2) ...
Setting up gnupg-l10n (2.2.19-3ubuntu2.1) ...
Setting up librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2build1) ...
Setting up libk5crypto3:amd64 (1.17-6ubuntu4.1) ...
Setting up libsasl2-2:amd64 (2.1.27+dfsg-2) ...
Setting up libroken18-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libkrb5-3:amd64 (1.17-6ubuntu4.1) ...
Setting up openssl (1.1.1f-1ubuntu2.10) ...
Setting up readline-common (8.0-4) ...
Setting up publicsuffix (20200303.0012-1) ...
Setting up libheimbase1-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up pinentry-curses (1.1.0-3build1) ...
Setting up libasn1-8-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libreadline8:amd64 (8.0-4) ...
Setting up libhcrypto4-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up ca-certificates (20210119~20.04.2) ...
debconf: unable to initialize frontend: Dialog
debconf: (TERM is not set, so the dialog frontend is not usable.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.30.0 /usr/local/share/perl/5.30.0 /usr/lib/x86_64-linux-gnu/perl5/5.30 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.30 /usr/share/perl/5.30 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 7.)
debconf: falling back to frontend: Teletype
Updating certificates in /etc/ssl/certs...
128 added, 0 removed; done.
Setting up libwind0-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libgssapi-krb5-2:amd64 (1.17-6ubuntu4.1) ...
Setting up libssh-4:amd64 (0.9.3-2ubuntu2.2) ...
Setting up gpgconf (2.2.19-3ubuntu2.1) ...
Setting up gpg (2.2.19-3ubuntu2.1) ...
Setting up gnupg-utils (2.2.19-3ubuntu2.1) ...
Setting up gpg-agent (2.2.19-3ubuntu2.1) ...
Setting up libhx509-5-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up gpgsm (2.2.19-3ubuntu2.1) ...
Setting up gpg-wks-server (2.2.19-3ubuntu2.1) ...
Setting up libkrb5-26-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libheimntlm0-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libgssapi3-heimdal:amd64 (7.7.0+dfsg-1ubuntu1) ...
Setting up libldap-2.4-2:amd64 (2.4.49+dfsg-2ubuntu1.8) ...
Setting up dirmngr (2.2.19-3ubuntu2.1) ...
Setting up libcurl4:amd64 (7.68.0-1ubuntu2.7) ...
Setting up curl (7.68.0-1ubuntu2.7) ...
Setting up gpg-wks-client (2.2.19-3ubuntu2.1) ...
Setting up gnupg (2.2.19-3ubuntu2.1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
Processing triggers for ca-certificates (20210119~20.04.2) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Removing intermediate container e5864b0382ee
 ---> 6cdd68ece3a1
Step 8/12 : RUN curl -s https://packages-dev.wazuh.com/key/GPG-KEY-WAZUH | apt-key add - &&     echo "deb https://packages-dev.wazuh.com/trash/apt/ unstable main" | tee -a /etc/apt/sources.list.d/wazuh.list &&     apt-get update &&     apt-get install wazuh-indexer
 ---> Running in d1a71668231a
Warning: apt-key output should not be parsed (stdout is not a terminal)
OK
deb https://packages-dev.wazuh.com/trash/apt/ unstable main
Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease
Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Get:4 https://packages-dev.wazuh.com/trash/apt unstable InRelease [12.7 kB]
Hit:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Get:6 https://packages-dev.wazuh.com/trash/apt unstable/main amd64 Packages [4312 B]
Fetched 17.0 kB in 2s (7926 B/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  wazuh-indexer
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 359 MB of archives.
After this operation, 639 MB of additional disk space will be used.
Get:1 https://packages-dev.wazuh.com/trash/apt unstable/main amd64 wazuh-indexer amd64 4.3.0-0.0.0.todelete [359 MB]
Get:1 https://packages-dev.wazuh.com/trash/apt unstable/main amd64 wazuh-indexer amd64 4.3.0-0.0.0.todelete [359 MB]
Get:1 https://packages-dev.wazuh.com/trash/apt unstable/main amd64 wazuh-indexer amd64 4.3.0-0.0.0.todelete [359 MB]
Get:1 https://packages-dev.wazuh.com/trash/apt unstable/main amd64 wazuh-indexer amd64 4.3.0-0.0.0.todelete [359 MB]
Get:1 https://packages-dev.wazuh.com/trash/apt unstable/main amd64 wazuh-indexer amd64 4.3.0-0.0.0.todelete [359 MB]
Get:1 https://packages-dev.wazuh.com/trash/apt unstable/main amd64 wazuh-indexer amd64 4.3.0-0.0.0.todelete [359 MB]
Get:1 https://packages-dev.wazuh.com/trash/apt unstable/main amd64 wazuh-indexer amd64 4.3.0-0.0.0.todelete [359 MB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 24.4 MB in 2min 44s (149 kB/s)
Selecting previously unselected package wazuh-indexer.
(Reading database ... 4895 files and directories currently installed.)
Preparing to unpack .../wazuh-indexer_4.3.0-0.0.0.todelete_amd64.deb ...
Creating wazuh-indexer group... OK
Creating wazuh-indexer user... OK
Unpacking wazuh-indexer (4.3.0-0.0.0.todelete) ...
Setting up wazuh-indexer (4.3.0-0.0.0.todelete) ...
dpkg: error processing package wazuh-indexer (--configure):
 installed wazuh-indexer package post-installation script subprocess returned error exit status 255
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
Errors were encountered while processing:
 wazuh-indexer
E: Sub-process /usr/bin/dpkg returned an error code (1)
The command '/bin/sh -c curl -s https://packages-dev.wazuh.com/key/GPG-KEY-WAZUH | apt-key add - &&     echo "deb https://packages-dev.wazuh.com/trash/apt/ unstable main" | tee -a /etc/apt/sources.list.d/wazuh.list &&     apt-get update &&     apt-get install wazuh-indexer' returned a non-zero code: 100
$

Tried installing on a Ubuntu Focal VM but shows the same error.

The Dockerfile was created for the build of the wazuh-dashboard image, but a complete build of it could not be done.

vcerenu commented 2 years ago

We proceeded to build the wazuh-indexer image for ubuntu, but got errors:

Step 8/12 : RUN curl https://s3.amazonaws.com/warehouse.wazuh.com/stack/indexer/stable/wazuh-indexer_${WAZUH_VERSION}_amd64.deb --output wazuh-indexer_${WAZUH_VERSION}_amd64.deb &&     dpkg -i wazuh-indexer_${WAZUH_VERSION}_amd64.deb
 ---> Running in 6069804ba072
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  343M  100  343M    0     0  5267k      0  0:01:06  0:01:06 --:--:-- 5345k
Selecting previously unselected package wazuh-indexer.
(Reading database ... 4660 files and directories currently installed.)
Preparing to unpack wazuh-indexer_4.3.0-1_amd64.deb ...
Creating wazuh-indexer group... OK
Creating wazuh-indexer user... OK
Unpacking wazuh-indexer (4.3.0-1) ...
Setting up wazuh-indexer (4.3.0-1) ...
dpkg: error processing package wazuh-indexer (--install):
 installed wazuh-indexer package post-installation script subprocess returned error exit status 2
Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
Errors were encountered while processing:
 wazuh-indexer
The command '/bin/sh -c curl https://s3.amazonaws.com/warehouse.wazuh.com/stack/indexer/stable/wazuh-indexer_${WAZUH_VERSION}_amd64.deb --output wazuh-indexer_${WAZUH_VERSION}_amd64.deb &&     dpkg -i wazuh-indexer_${WAZUH_VERSION}_amd64.deb' returned a non-zero code: 1

Tried to install the same package on a VM and got the same error.

The image was created based on the Centos:8 image and there were no errors in the creation:

$ docker build -f Dockerfile_centos -t indexer .
Sending build context to Docker daemon   12.8kB
Step 1/13 : FROM centos:8
 ---> 5d0da3dc9764
Step 2/13 : ARG FILEBEAT_CHANNEL=filebeat-oss
 ---> Using cache
 ---> 595bbc806385
Step 3/13 : ARG FILEBEAT_VERSION=7.10.2
 ---> Using cache
 ---> c1ac7e474ce2
Step 4/13 : ARG WAZUH_VERSION=4.3.0-1
 ---> Using cache
 ---> 23015db8dc3b
Step 5/13 : ARG TEMPLATE_VERSION="master"
 ---> Using cache
 ---> 2c06f6eca54c
Step 6/13 : ARG WAZUH_FILEBEAT_MODULE="wazuh-filebeat-0.1.tar.gz"
 ---> Using cache
 ---> b7c816cd86dc
Step 7/13 : USER root
 ---> Using cache
 ---> a09c85e4006a
Step 8/13 : RUN yum install initscripts -y
 ---> Using cache
 ---> 7a3aa7ffcee5
Step 9/13 : RUN curl https://s3.amazonaws.com/warehouse.wazuh.com/stack/indexer/stable/wazuh-indexer-${WAZUH_VERSION}.x86_64.rpm --output wazuh-indexer-${WAZUH_VERSION}.x86_64.rpm &&     rpm -i wazuh-indexer-${WAZUH_VERSION}.x86_64.rpm
 ---> Running in ff73d0c81fc1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  361M  100  361M    0     0  4918k      0  0:01:15  0:01:15 --:--:--  9.8M
/var/tmp/rpm-tmp.2zOMYh: line 16: sudo: command not found
Removing intermediate container ff73d0c81fc1
 ---> c675e54447a8
Step 10/13 : COPY config/entrypoint.sh /
 ---> a4bf5d0b0e68
Step 11/13 : RUN chmod 700 /entrypoint.sh
 ---> Running in 79df0d4c3103
Removing intermediate container 79df0d4c3103
 ---> bc5f7e427a5c
Step 12/13 : EXPOSE 9700
 ---> Running in 7c353b10eb31
Removing intermediate container 7c353b10eb31
 ---> 838a514ab152
Step 13/13 : ENTRYPOINT [ "/entrypoint.sh" ]
 ---> Running in 9381f6340606
Removing intermediate container 9381f6340606
 ---> 225bdbd63010
Successfully built 225bdbd63010
Successfully tagged indexer:latest
$

After creating the images, I proceeded to modify the docker-compose.yml to perform a deployment test, but I did not have satisfactory results with the deployment of wazuh-indexer:

$ docker logs wazuh-docker_elasticsearch_1
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
Starting wazuh-indexer: [  OK  ]
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by io.protostuff.runtime.PolymorphicThrowableSchema (file:/usr/share/wazuh-indexer/plugins/opensearch-anomaly-detection/protostuff-runtime-1.7.4.jar) to field java.lang.Throwable.cause
WARNING: Please consider reporting this to the maintainers of io.protostuff.runtime.PolymorphicThrowableSchema
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
vcerenu@vcerenu-VirtualBox:~/Repositorios/wazuh-docker$ curl -k -u admin:admin https://0.0.0.0:9700
OpenSearch Security not initialized.
$
vcerenu commented 2 years ago

The wazuh-indexer and wazuh-dashboard entrypoints were modified, adding necessary parameters for the start of the services.

The packages used for the installation of both tools have been updated.

The docker-compose.yml file was nodified to be able to build a stack of wazuh-manager, wazuh-indexer and wazuh-dashboard.

After several errors, it was possible to leave a stack started without errors.

wazuh-indexer:

$ docker logs wazuh-docker_elasticsearch_1 -f
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
Starting wazuh-indexer: [  OK  ]
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by io.protostuff.runtime.PolymorphicThrowableSchema (file:/usr/share/wazuh-indexer/plugins/opensearch-anomaly-detection/protostuff-runtime-1.7.4.jar) to field java.lang.Throwable.cause
WARNING: Please consider reporting this to the maintainers of io.protostuff.runtime.PolymorphicThrowableSchema
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Security Admin v7
Will connect to localhost:9800 ... done
Connected as CN=admin,OU=Docu,O=Wazuh,L=California,C=US
OpenSearch Version: 1.2.3
OpenSearch Security Version: 1.2.3.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: wazuh-cluster
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index does not exists, attempt to create it ... done (0-all replicas)
Populate config from /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/
Will update '_doc/config' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/config.yml 
   SUCC: Configuration for 'config' created or updated
Will update '_doc/roles' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/roles.yml 
   SUCC: Configuration for 'roles' created or updated
Will update '_doc/rolesmapping' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/roles_mapping.yml 
   SUCC: Configuration for 'rolesmapping' created or updated
Will update '_doc/internalusers' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/internal_users.yml 
   SUCC: Configuration for 'internalusers' created or updated
Will update '_doc/actiongroups' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/action_groups.yml 
   SUCC: Configuration for 'actiongroups' created or updated
Will update '_doc/tenants' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/tenants.yml 
   SUCC: Configuration for 'tenants' created or updated
Will update '_doc/nodesdn' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/nodes_dn.yml 
   SUCC: Configuration for 'nodesdn' created or updated
Will update '_doc/whitelist' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/whitelist.yml 
   SUCC: Configuration for 'whitelist' created or updated
Will update '_doc/audit' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/audit.yml 
   SUCC: Configuration for 'audit' created or updated
Done with success
[2022-01-21T18:26:23,341][INFO ][o.o.s.a.i.AuditLogImpl   ] [node-1] Auditing of internal configuration is enabled.
[2022-01-21T18:26:23,341][INFO ][o.o.s.a.i.AuditLogImpl   ] [node-1] Auditing only metadata information for read request is enabled.
[2022-01-21T18:26:23,341][INFO ][o.o.s.a.i.AuditLogImpl   ] [node-1] Auditing will watch {} for read requests.
[2022-01-21T18:26:23,341][INFO ][o.o.s.a.i.AuditLogImpl   ] [node-1] Auditing read operation requests from kibanaserver users is disabled.
[2022-01-21T18:26:23,341][INFO ][o.o.s.a.i.AuditLogImpl   ] [node-1] Auditing only metadata information for write request is enabled.
[2022-01-21T18:26:23,342][INFO ][o.o.s.a.i.AuditLogImpl   ] [node-1] Auditing diffs for write requests is disabled.
[2022-01-21T18:26:23,342][INFO ][o.o.s.a.i.AuditLogImpl   ] [node-1] Auditing write operation requests from kibanaserver users is disabled.
[2022-01-21T18:26:23,342][INFO ][o.o.s.a.i.AuditLogImpl   ] [node-1] Auditing will watch <NONE> for write requests.
[2022-01-21T18:26:23,342][INFO ][o.o.s.a.i.AuditLogImpl   ] [node-1] .opendistro_security is used as internal security index.
[2022-01-21T18:26:23,342][INFO ][o.o.s.a.i.AuditLogImpl   ] [node-1] Internal index used for posting audit logs is null
[2022-01-21T18:26:24,296][INFO ][o.o.s.c.ConfigurationRepository] [node-1] Hot-reloading of audit configuration is enabled
[2022-01-21T18:26:24,296][INFO ][o.o.s.c.ConfigurationRepository] [node-1] Node 'node-1' initialized
[2022-01-21T18:26:25,920][INFO ][o.o.c.m.MetadataCreateIndexService] [node-1] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1]
[2022-01-21T18:26:25,921][INFO ][o.o.c.r.a.AllocationService] [node-1] updating number_of_replicas to [0] for indices [.kibana_1]
[2022-01-21T18:26:25,992][INFO ][o.o.c.r.a.AllocationService] [node-1] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.kibana_1][0]]]).
[2022-01-21T18:26:31,947][INFO ][o.o.c.m.MetadataCreateIndexService] [node-1] [security-auditlog-2022.01.21] creating index, cause [auto(bulk api)], templates [], shards [1]/[1]
[2022-01-21T18:26:32,025][INFO ][o.o.c.m.MetadataIndexTemplateService] [node-1] adding template [wazuh] for index patterns [wazuh-alerts-4.x-*, wazuh-archives-4.x-*]
[2022-01-21T18:26:32,103][INFO ][o.o.c.m.MetadataMappingService] [node-1] [security-auditlog-2022.01.21/rw1ok2UjRpGRPbvfqDheCQ] create_mapping [_doc]
[2022-01-21T18:26:32,186][INFO ][o.o.c.m.MetadataCreateIndexService] [node-1] [wazuh-alerts-4.x-2022.01.21] creating index, cause [auto(bulk api)], templates [wazuh], shards [3]/[0]
[2022-01-21T18:26:32,267][INFO ][o.o.c.m.MetadataMappingService] [node-1] [security-auditlog-2022.01.21/rw1ok2UjRpGRPbvfqDheCQ] update_mapping [_doc]
[2022-01-21T18:26:32,423][INFO ][o.o.c.m.MetadataMappingService] [node-1] [wazuh-alerts-4.x-2022.01.21/OlCHJtwwTv-2jAShWp3O-Q] update_mapping [_doc]
[2022-01-21T18:26:32,490][INFO ][o.o.c.m.MetadataMappingService] [node-1] [security-auditlog-2022.01.21/rw1ok2UjRpGRPbvfqDheCQ] update_mapping [_doc]
[2022-01-21T18:26:32,514][INFO ][o.o.c.m.MetadataMappingService] [node-1] [wazuh-alerts-4.x-2022.01.21/OlCHJtwwTv-2jAShWp3O-Q] update_mapping [_doc]
[2022-01-21T18:26:32,693][INFO ][o.o.c.m.MetadataMappingService] [node-1] [wazuh-alerts-4.x-2022.01.21/OlCHJtwwTv-2jAShWp3O-Q] update_mapping [_doc]
[2022-01-21T18:27:08,247][INFO ][o.o.i.i.ManagedIndexCoordinator] [node-1] Performing move cluster state metadata.
[2022-01-21T18:28:08,250][INFO ][o.o.i.i.ManagedIndexCoordinator] [node-1] Performing move cluster state metadata.
[2022-01-21T18:29:08,253][INFO ][o.o.i.i.ManagedIndexCoordinator] [node-1] Performing move cluster state metadata.
[2022-01-21T18:30:08,255][INFO ][o.o.i.i.ManagedIndexCoordinator] [node-1] Performing move cluster state metadata.
[2022-01-21T18:30:08,255][INFO ][o.o.i.i.MetadataService  ] [node-1] Move Metadata succeed, set finish flag to true. Indices failed to get indexed: {}
[2022-01-21T18:31:07,980][INFO ][o.o.j.s.JobSweeper       ] [node-1] Running full sweep
[2022-01-21T18:31:08,256][INFO ][o.o.i.i.ManagedIndexCoordinator] [node-1] Cancel background move metadata process.
[2022-01-21T18:31:08,256][INFO ][o.o.i.i.ManagedIndexCoordinator] [node-1] Performing move cluster state metadata.
[2022-01-21T18:31:08,256][INFO ][o.o.i.i.MetadataService  ] [node-1] Move metadata has finished.

wazuh-dashboard:

$ docker logs wazuh-docker_kibana_1 -f
wazuh-dashboard started
{"type":"log","@timestamp":"2022-01-21T18:26:00Z","tags":["info","plugins-system"],"pid":14,"message":"Setting up [44] plugins: [usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,alertingDashboards,share,opensearchUiShared,legacyExport,embeddable,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,dashboard,visualizations,visTypeTimeline,timeline,visTypeVega,visTypeTable,visTypeMarkdown,tileMap,regionMap,inputControlVis,visualize,ganttChartDashboards,securityDashboards,reportsDashboards,indexManagementDashboards,anomalyDetectionDashboards,queryWorkbenchDashboards,charts,visTypeTimeseries,visTypeTagcloud,visTypeVislib,visTypeMetric,discover,savedObjectsManagement,observabilityDashboards,bfetch]"}
{"type":"log","@timestamp":"2022-01-21T18:26:00Z","tags":["info","savedobjects-service"],"pid":14,"message":"Waiting until all OpenSearch nodes are compatible with OpenSearch Dashboards before starting saved objects migrations..."}
{"type":"log","@timestamp":"2022-01-21T18:26:00Z","tags":["error","opensearch","data"],"pid":14,"message":"[ConnectionError]: connect ECONNREFUSED 192.168.96.2:9700"}
{"type":"log","@timestamp":"2022-01-21T18:26:00Z","tags":["error","savedobjects-service"],"pid":14,"message":"Unable to retrieve version information from OpenSearch nodes."}
{"type":"log","@timestamp":"2022-01-21T18:26:03Z","tags":["error","opensearch","data"],"pid":14,"message":"[ConnectionError]: connect ECONNREFUSED 192.168.96.2:9700"}
{"type":"log","@timestamp":"2022-01-21T18:26:05Z","tags":["error","opensearch","data"],"pid":14,"message":"[ConnectionError]: connect ECONNREFUSED 192.168.96.2:9700"}
{"type":"log","@timestamp":"2022-01-21T18:26:08Z","tags":["error","opensearch","data"],"pid":14,"message":"[ConnectionError]: connect ECONNREFUSED 192.168.96.2:9700"}
{"type":"log","@timestamp":"2022-01-21T18:26:10Z","tags":["error","opensearch","data"],"pid":14,"message":"[ResponseError]: Response Error"}
{"type":"log","@timestamp":"2022-01-21T18:26:13Z","tags":["error","opensearch","data"],"pid":14,"message":"[ResponseError]: Response Error"}
{"type":"log","@timestamp":"2022-01-21T18:26:15Z","tags":["error","opensearch","data"],"pid":14,"message":"[ResponseError]: Response Error"}
{"type":"log","@timestamp":"2022-01-21T18:26:18Z","tags":["error","opensearch","data"],"pid":14,"message":"[ResponseError]: Response Error"}
{"type":"log","@timestamp":"2022-01-21T18:26:20Z","tags":["error","opensearch","data"],"pid":14,"message":"[ResponseError]: Response Error"}
{"type":"log","@timestamp":"2022-01-21T18:26:23Z","tags":["error","opensearch","data"],"pid":14,"message":"[ResponseError]: Response Error"}
{"type":"log","@timestamp":"2022-01-21T18:26:25Z","tags":["info","savedobjects-service"],"pid":14,"message":"Starting saved objects migrations"}
{"type":"log","@timestamp":"2022-01-21T18:26:25Z","tags":["info","savedobjects-service"],"pid":14,"message":"Creating index .kibana_1."}
{"type":"log","@timestamp":"2022-01-21T18:26:26Z","tags":["info","savedobjects-service"],"pid":14,"message":"Pointing alias .kibana to .kibana_1."}
{"type":"log","@timestamp":"2022-01-21T18:26:26Z","tags":["info","savedobjects-service"],"pid":14,"message":"Finished in 189ms."}
{"type":"log","@timestamp":"2022-01-21T18:26:26Z","tags":["info","plugins-system"],"pid":14,"message":"Starting [44] plugins: [usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,alertingDashboards,share,opensearchUiShared,legacyExport,embeddable,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,dashboard,visualizations,visTypeTimeline,timeline,visTypeVega,visTypeTable,visTypeMarkdown,tileMap,regionMap,inputControlVis,visualize,ganttChartDashboards,securityDashboards,reportsDashboards,indexManagementDashboards,anomalyDetectionDashboards,queryWorkbenchDashboards,charts,visTypeTimeseries,visTypeTagcloud,visTypeVislib,visTypeMetric,discover,savedObjectsManagement,observabilityDashboards,bfetch]"}
{"type":"log","@timestamp":"2022-01-21T18:26:26Z","tags":["listening","info"],"pid":14,"message":"Server running at https://0.0.0.0:443"}
{"type":"log","@timestamp":"2022-01-21T18:26:26Z","tags":["info","http","server","OpenSearchDashboards"],"pid":14,"message":"http server running at https://0.0.0.0:443"}

wazuh-manager:

$ docker logs wazuh-docker_wazuh_1 -f
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 0-wazuh-init: executing... 
Installing /var/ossec/api/configuration
Installing /var/ossec/etc
Installing /var/ossec/logs
Installing /var/ossec/queue
Installing /var/ossec/agentless
Installing /var/ossec/var/multigroups
Installing /var/ossec/integrations
Installing /var/ossec/active-response/bin
Installing /var/ossec/wodles
Installing /etc/filebeat
Updating /var/ossec/etc/internal_options.conf
Updating /var/ossec/integrations/pagerduty
Updating /var/ossec/integrations/slack
Updating /var/ossec/integrations/slack.py
Updating /var/ossec/integrations/virustotal
Updating /var/ossec/integrations/virustotal.py
Updating /var/ossec/active-response/bin/default-firewall-drop
Updating /var/ossec/active-response/bin/disable-account
Updating /var/ossec/active-response/bin/firewalld-drop
Updating /var/ossec/active-response/bin/firewall-drop
Updating /var/ossec/active-response/bin/host-deny
Updating /var/ossec/active-response/bin/ip-customblock
Updating /var/ossec/active-response/bin/ipfw
Updating /var/ossec/active-response/bin/kaspersky.py
Updating /var/ossec/active-response/bin/kaspersky
Updating /var/ossec/active-response/bin/npf
Updating /var/ossec/active-response/bin/wazuh-slack
Updating /var/ossec/active-response/bin/pf
Updating /var/ossec/active-response/bin/restart-wazuh
Updating /var/ossec/active-response/bin/restart.sh
Updating /var/ossec/active-response/bin/route-null
Updating /var/ossec/agentless/sshlogin.exp
Updating /var/ossec/agentless/ssh_pixconfig_diff
Updating /var/ossec/agentless/ssh_asa-fwsmconfig_diff
Updating /var/ossec/agentless/ssh_integrity_check_bsd
Updating /var/ossec/agentless/main.exp
Updating /var/ossec/agentless/su.exp
Updating /var/ossec/agentless/ssh_integrity_check_linux
Updating /var/ossec/agentless/register_host.sh
Updating /var/ossec/agentless/ssh_generic_diff
Updating /var/ossec/agentless/ssh_foundry_diff
Updating /var/ossec/agentless/ssh_nopass.exp
Updating /var/ossec/agentless/ssh.exp
Updating /var/ossec/wodles/aws/aws-s3
Updating /var/ossec/wodles/aws/aws-s3.py
Updating /var/ossec/wodles/azure/azure-logs
Updating /var/ossec/wodles/azure/azure-logs.py
Updating /var/ossec/wodles/docker/DockerListener
Updating /var/ossec/wodles/docker/DockerListener.py
Updating /var/ossec/wodles/gcloud/gcloud
Updating /var/ossec/wodles/gcloud/gcloud.py
Updating /var/ossec/wodles/gcloud/integration.py
Updating /var/ossec/wodles/gcloud/tools.py
Updating /var/ossec/wodles/utils.py
No Wazuh configuration files to mount...
[cont-init.d] 0-wazuh-init: exited 0.
[cont-init.d] 1-config-filebeat: executing... 
Customize Elasticsearch ouput IP
Configuring username.
Configuring password.
Configuring SSL verification mode.
[cont-init.d] 1-config-filebeat: exited 0.
[cont-init.d] 2-manager: executing... 
Starting Wazuh v4.2.5...
Started wazuh-apid...
Started wazuh-csyslogd...
Started wazuh-dbd...
2022/01/21 18:26:05 wazuh-integratord: INFO: Remote integrations not configured. Clean exit.
Started wazuh-integratord...
Started wazuh-agentlessd...
Started wazuh-authd...
Started wazuh-db...
Started wazuh-execd...
Started wazuh-analysisd...
Started wazuh-syscheckd...
Started wazuh-remoted...
Started wazuh-logcollector...
Started wazuh-monitord...
Started wazuh-modulesd...
Completed.
[cont-init.d] 2-manager: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
2022/01/21 18:26:12 wazuh-modulesd:download: INFO: Module started.
2022/01/21 18:26:12 wazuh-modulesd:task-manager: INFO: (8200): Module Task Manager started.
2022/01/21 18:26:12 sca: INFO: Module started.
2022/01/21 18:26:12 sca: INFO: Loaded policy '/var/ossec/ruleset/sca/cis_centos7_linux.yml'
2022/01/21 18:26:12 sca: INFO: Starting Security Configuration Assessment scan.
2022/01/21 18:26:12 wazuh-modulesd:control: INFO: Starting control thread.
2022/01/21 18:26:12 wazuh-modulesd:syscollector: INFO: Module started.
2022/01/21 18:26:12 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2022/01/21 18:26:12 sca: INFO: Starting evaluation of policy: '/var/ossec/ruleset/sca/cis_centos7_linux.yml'
2022/01/21 18:26:12 wazuh-modulesd:syscollector: INFO: Evaluation finished.
starting Filebeat
2022-01-21T18:26:15.618Z    INFO    instance/beat.go:645    Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2022-01-21T18:26:15.621Z    INFO    instance/beat.go:653    Beat ID: 32c77775-f804-456d-9932-4e00c74c9116
2022-01-21T18:26:15.622Z    INFO    [seccomp]   seccomp/seccomp.go:124  Syscall filter successfully installed
2022-01-21T18:26:15.622Z    INFO    [beat]  instance/beat.go:981    Beat info   {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "32c77775-f804-456d-9932-4e00c74c9116"}}}
2022-01-21T18:26:15.622Z    INFO    [beat]  instance/beat.go:990    Build info  {"system_info": {"build": {"commit": "aacf9ecd9c494aa0908f61fbca82c906b16562a8", "libbeat": "7.10.2", "time": "2021-01-12T22:10:33.000Z", "version": "7.10.2"}}}
2022-01-21T18:26:15.622Z    INFO    [beat]  instance/beat.go:993    Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":4,"version":"go1.14.12"}}}
2022-01-21T18:26:15.623Z    INFO    [beat]  instance/beat.go:997    Host info   {"system_info": {"host": {"architecture":"x86_64","boot_time":"2022-01-21T13:17:58Z","containerized":true,"name":"wazuh-manager","ip":["127.0.0.1/8","192.168.96.3/20"],"kernel_version":"5.13.0-27-generic","mac":["02:42:c0:a8:60:03"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":9,"patch":2009,"codename":"Core"},"timezone":"UTC","timezone_offset_sec":0,"id":"acfc28e237eff049a14e0ac9b568a789"}}}
2022-01-21T18:26:15.623Z    INFO    [beat]  instance/beat.go:1026   Process info    {"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/run/s6/services/filebeat", "exe": "/usr/share/filebeat/bin/filebeat", "name": "filebeat", "pid": 1028, "ppid": 1025, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2022-01-21T18:26:15.190Z"}}}
2022-01-21T18:26:15.623Z    INFO    instance/beat.go:299    Setup Beat: filebeat; Version: 7.10.2
2022-01-21T18:26:15.623Z    INFO    eslegclient/connection.go:99    elasticsearch url: https://elasticsearch:9700
2022-01-21T18:26:15.623Z    WARN    [tls]   tlscommon/tls_config.go:93  SSL/TLS verifications disabled.
2022-01-21T18:26:15.623Z    INFO    [publisher] pipeline/module.go:113  Beat name: wazuh-manager
2022-01-21T18:26:15.624Z    INFO    beater/filebeat.go:117  Enabled modules/filesets: wazuh (alerts),  ()
2022-01-21T18:26:15.624Z    INFO    instance/beat.go:455    filebeat start running.
2022-01-21T18:26:15.625Z    INFO    [monitoring]    log/log.go:118  Starting metrics logging every 30s
2022-01-21T18:26:15.629Z    INFO    memlog/store.go:119 Loading data file of '/var/lib/filebeat/registry/filebeat' succeeded. Active transaction id=0
2022-01-21T18:26:15.629Z    INFO    memlog/store.go:124 Finished loading transaction log file for '/var/lib/filebeat/registry/filebeat'. Active transaction id=0
2022-01-21T18:26:15.629Z    INFO    [registrar] registrar/registrar.go:109  States Loaded from registrar: 0
2022-01-21T18:26:15.629Z    INFO    [crawler]   beater/crawler.go:71    Loading Inputs: 1
2022-01-21T18:26:15.629Z    INFO    log/input.go:157    Configured paths: [/var/ossec/logs/alerts/alerts.json]
2022-01-21T18:26:15.629Z    INFO    [crawler]   beater/crawler.go:141   Starting input (ID: 9132358592892857476)
2022-01-21T18:26:15.629Z    INFO    [crawler]   beater/crawler.go:108   Loading and starting Inputs completed. Enabled inputs: 1
2022-01-21T18:26:15.629Z    INFO    log/harvester.go:302    Harvester started for file: /var/ossec/logs/alerts/alerts.json
2022/01/21 18:26:21 sca: INFO: Evaluation finished for policy '/var/ossec/ruleset/sca/cis_centos7_linux.yml'
2022/01/21 18:26:21 sca: INFO: Security Configuration Assessment scan finished. Duration: 9 seconds.
2022/01/21 18:26:24 rootcheck: INFO: Ending rootcheck scan.
2022-01-21T18:26:31.634Z    INFO    [publisher_pipeline_output] pipeline/output.go:143  Connecting to backoff(elasticsearch(https://elasticsearch:9700))
2022-01-21T18:26:31.634Z    INFO    [publisher] pipeline/retry.go:219   retryer: send unwait signal to consumer
2022-01-21T18:26:31.634Z    INFO    [publisher] pipeline/retry.go:223     done
2022-01-21T18:26:31.636Z    WARN    [tls]   tlscommon/tls_config.go:93  SSL/TLS verifications disabled.
2022-01-21T18:26:31.845Z    INFO    [esclientleg]   eslegclient/connection.go:314   Attempting to connect to Elasticsearch version 7.10.2
2022-01-21T18:26:31.847Z    INFO    [esclientleg]   eslegclient/connection.go:314   Attempting to connect to Elasticsearch version 7.10.2
2022-01-21T18:26:31.907Z    INFO    fileset/pipelines.go:143    Elasticsearch pipeline with ID 'filebeat-7.10.2-wazuh-alerts-pipeline' loaded
2022-01-21T18:26:31.910Z    INFO    template/load.go:183    Existing template will be overwritten, as overwrite is enabled.
2022-01-21T18:26:31.911Z    INFO    template/load.go:117    Try loading template wazuh to Elasticsearch
2022-01-21T18:26:32.055Z    INFO    template/load.go:109    template with name 'wazuh' loaded.
2022-01-21T18:26:32.055Z    INFO    [index-management]  idxmgmt/std.go:298  Loaded index template.
2022-01-21T18:26:32.057Z    INFO    [publisher_pipeline_output] pipeline/output.go:151  Connection to backoff(elasticsearch(https://elasticsearch:9700)) established

Wazuh app home screen: image

image

vcerenu commented 2 years ago

A new Docker image was created to be able to generate the certificates with the wazuh-indexer image on the directory in which we are going to use to mount the keys on the stack containers generated by the production-cluster.yml script.

Modified production-clsuter.yml script to mount volumes over directories used by wazuh-indexer.

When raising the cluster it generates the following error:

elasticsearch_1    | uncaught exception in thread [main]
elasticsearch_1    | java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
elasticsearch_1    | Likely root cause: OpenSearchException[plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.]
elasticsearch_1    |    at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:422)
elasticsearch_1    |    at org.opensearch.security.ssl.DefaultSecurityKeyStore.initSSLConfig(DefaultSecurityKeyStore.java:258)
elasticsearch_1    |    at org.opensearch.security.ssl.DefaultSecurityKeyStore.<init>(DefaultSecurityKeyStore.java:179)
elasticsearch_1    |    at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.<init>(OpenSearchSecuritySSLPlugin.java:218)
elasticsearch_1    |    at org.opensearch.security.OpenSearchSecurityPlugin.<init>(OpenSearchSecurityPlugin.java:252)
elasticsearch_1    |    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
elasticsearch_1    |    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64)
elasticsearch_1    |    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
elasticsearch_1    |    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
elasticsearch_1    |    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
elasticsearch_1    |    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:781)
elasticsearch_1    |    at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:726)
elasticsearch_1    |    at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:528)
elasticsearch_1    |    at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:194)
elasticsearch_1    |    at org.opensearch.node.Node.<init>(Node.java:396)
elasticsearch_1    |    at org.opensearch.node.Node.<init>(Node.java:319)
elasticsearch_1    |    at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
elasticsearch_1    |    at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
elasticsearch_1    |    at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412)
elasticsearch_1    |    at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178)
elasticsearch_1    |    at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169)
elasticsearch_1    |    at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100)
elasticsearch_1    |    at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
elasticsearch_1    |    at org.opensearch.cli.Command.main(Command.java:101)
elasticsearch_1    |    at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135)
elasticsearch_1    |    at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101)
elasticsearch_1    | For complete error details, refer to the log at /usr/share/wazuh-indexer/logs/wazuh-cluster.log

We continue to investigate the error and the necessary configuration for the cluster.

vcerenu commented 2 years ago

The configuration of the yaml file sent to each of the nodes was revised to take the necessary parameter values for wazuh-indexer. They are located inside the production-cluster/elastic_opendistro folder.

When performing the stack startup test with a cluster of 3 wazuh-indexer nodes, it logs the following errors:

System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
Starting wazuh-indexer: [  OK  ]
wazuh-indexer (pid  198) is running...
uncaught exception in thread [main]
java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
Likely root cause: OpenSearchException[plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.]
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:422)
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.initSSLConfig(DefaultSecurityKeyStore.java:258)
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.<init>(DefaultSecurityKeyStore.java:179)
    at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.<init>(OpenSearchSecuritySSLPlugin.java:218)
    at org.opensearch.security.OpenSearchSecurityPlugin.<init>(OpenSearchSecurityPlugin.java:252)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:781)
    at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:726)
    at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:528)
    at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:194)
    at org.opensearch.node.Node.<init>(Node.java:396)
    at org.opensearch.node.Node.<init>(Node.java:319)
    at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242)
    at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242)
    at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412)
    at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178)
    at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169)
    at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100)
    at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
    at org.opensearch.cli.Command.main(Command.java:101)
    at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135)
    at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101)
For complete error details, refer to the log at /var/log/wazuh-indexer/opensearch.log
Security Admin v7
Will connect to localhost:9800
ERR: Seems there is no OpenSearch running on localhost:9800 - Will exit

Error details in /var/log/wazuh-indexer/opensearch.log file:

[2022-01-25T21:00:44,975][INFO ][o.o.n.Node               ] [node1] version[1.2.3], pid[201], build[rpm/8a529d77c7432bc45b005ac1c4ba3b2741b57d4a/2021-12-21T01:36:21.407473Z], OS[Linux/5.13.0-27-generic/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
[2022-01-25T21:00:45,013][INFO ][o.o.n.Node               ] [node1] JVM home [/usr/share/wazuh-indexer/jdk], using bundled JDK [true]
[2022-01-25T21:00:45,014][INFO ][o.o.n.Node               ] [node1] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-3583472589606520261, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/usr/share/wazuh-indexer, -Dopensearch.path.conf=/etc/wazuh-indexer, -Dopensearch.distribution.type=rpm, -Dopensearch.bundled_jdk=true]
[2022-01-25T21:00:50,927][INFO ][o.o.s.s.t.SSLConfig      ] [node1] SSL dual mode is disabled
[2022-01-25T21:00:50,928][INFO ][o.o.s.OpenSearchSecurityPlugin] [node1] OpenSearch Config path is /etc/wazuh-indexer
[2022-01-25T21:01:05,892][INFO ][o.o.n.Node               ] [node1] version[1.2.3], pid[198], build[rpm/8a529d77c7432bc45b005ac1c4ba3b2741b57d4a/2021-12-21T01:36:21.407473Z], OS[Linux/5.13.0-27-generic/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15.0.1/15.0.1+9]
[2022-01-25T21:01:05,895][INFO ][o.o.n.Node               ] [node1] JVM home [/usr/share/wazuh-indexer/jdk], using bundled JDK [true]
[2022-01-25T21:01:05,931][INFO ][o.o.n.Node               ] [node1] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-17364797749620451537, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/usr/share/wazuh-indexer, -Dopensearch.path.conf=/etc/wazuh-indexer, -Dopensearch.distribution.type=rpm, -Dopensearch.bundled_jdk=true]
[2022-01-25T21:01:09,650][INFO ][o.o.s.s.t.SSLConfig      ] [node1] SSL dual mode is disabled
[2022-01-25T21:01:09,659][INFO ][o.o.s.OpenSearchSecurityPlugin] [node1] OpenSearch Config path is /etc/wazuh-indexer
[2022-01-25T21:01:10,491][INFO ][o.o.s.s.DefaultSecurityKeyStore] [node1] JVM supports TLSv1.3
[2022-01-25T21:01:10,493][INFO ][o.o.s.s.DefaultSecurityKeyStore] [node1] Config directory is /etc/wazuh-indexer/, from there the key- and truststore files are resolved relatively
[2022-01-25T21:01:10,810][ERROR][o.o.b.Bootstrap          ] [node1] Exception
java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:790) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:726) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:528) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:194) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.node.Node.<init>(Node.java:396) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.node.Node.<init>(Node.java:319) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412) [opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178) [opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169) [opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100) [opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) [opensearch-cli-1.2.3.jar:1.2.3]
    at org.opensearch.cli.Command.main(Command.java:101) [opensearch-cli-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135) [opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101) [opensearch-1.2.3.jar:1.2.3]
Caused by: java.lang.reflect.InvocationTargetException
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
    at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
    at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:781) ~[opensearch-1.2.3.jar:1.2.3]
    ... 15 more
Caused by: org.opensearch.OpenSearchException: plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:422) ~[?:?]
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.initSSLConfig(DefaultSecurityKeyStore.java:258) ~[?:?]
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.<init>(DefaultSecurityKeyStore.java:179) ~[?:?]
    at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.<init>(OpenSearchSecuritySSLPlugin.java:218) ~[?:?]
    at org.opensearch.security.OpenSearchSecurityPlugin.<init>(OpenSearchSecurityPlugin.java:252) ~[?:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
    at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
    at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:781) ~[opensearch-1.2.3.jar:1.2.3]
    ... 15 more
[2022-01-25T21:01:10,867][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [node1] uncaught exception in thread [main]
org.opensearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
    at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:182) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:100) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) ~[opensearch-cli-1.2.3.jar:1.2.3]
    at org.opensearch.cli.Command.main(Command.java:101) ~[opensearch-cli-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101) ~[opensearch-1.2.3.jar:1.2.3]
Caused by: java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:790) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:726) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:528) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:194) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.node.Node.<init>(Node.java:396) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.node.Node.<init>(Node.java:319) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178) ~[opensearch-1.2.3.jar:1.2.3]
    ... 6 more
Caused by: java.lang.reflect.InvocationTargetException
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
    at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
    at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:781) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:726) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:528) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:194) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.node.Node.<init>(Node.java:396) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.node.Node.<init>(Node.java:319) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178) ~[opensearch-1.2.3.jar:1.2.3]
    ... 6 more
Caused by: org.opensearch.OpenSearchException: plugins.security.ssl.transport.keystore_filepath or plugins.security.ssl.transport.server.pemcert_filepath and plugins.security.ssl.transport.client.pemcert_filepath must be set if transport ssl is requested.
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.initTransportSSLConfig(DefaultSecurityKeyStore.java:422) ~[?:?]
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.initSSLConfig(DefaultSecurityKeyStore.java:258) ~[?:?]
    at org.opensearch.security.ssl.DefaultSecurityKeyStore.<init>(DefaultSecurityKeyStore.java:179) ~[?:?]
    at org.opensearch.security.ssl.OpenSearchSecuritySSLPlugin.<init>(OpenSearchSecuritySSLPlugin.java:218) ~[?:?]
    at org.opensearch.security.OpenSearchSecurityPlugin.<init>(OpenSearchSecurityPlugin.java:252) ~[?:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64) ~[?:?]
    at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
    at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
    at org.opensearch.plugins.PluginsService.loadPlugin(PluginsService.java:781) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.plugins.PluginsService.loadBundle(PluginsService.java:726) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.plugins.PluginsService.loadBundles(PluginsService.java:528) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.plugins.PluginsService.<init>(PluginsService.java:194) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.node.Node.<init>(Node.java:396) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.node.Node.<init>(Node.java:319) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:412) ~[opensearch-1.2.3.jar:1.2.3]
    at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:178) ~[opensearch-1.2.3.jar:1.2.3]
    ... 6 more

Tried different configurations inside the main node's opensearch.yml file but it always logged that error.

New certificates were created with the Opendistro script to verify if the problem could be caused by that issue, but it remains the same.

vcerenu commented 2 years ago

Changed the opensearch.yml scripts for each node, as the security script used by the Opendistro script was changed

At the time of lifting the cluster does not start and log the following:

wazuh-indexer_1 | Will connect to localhost:9800
wazuh-indexer_1 | ERR: Seems there is no OpenSearch running on localhost:9800 - Will exit
wazuh-indexer_1 | cat: /var/log/wazuh-indexer/opensearch.log: No such file or directory
wazuh-indexer_1 | [2022-01-26T19:50:47,843][INFO ][o.o.p.PluginsService ] [wazuh-indexer] loaded plugin [opensearch-knn]
wazuh-indexer_1 | [2022-01-26T19:50:47,858][INFO ][o.o.p.PluginsService ] [wazuh-indexer] loaded plugin [opensearch-observability]
wazuh-indexer_1 | [2022-01-26T19:50:47,858][INFO ][o.o.p.PluginsService ] [wazuh-indexer] loaded plugin [opensearch-performance-analyzer]
wazuh-indexer_1 | [2022-01-26T19:50:47,858][INFO ][o.o.p.PluginsService ] [wazuh-indexer] loaded plugin [opensearch-reports-scheduler]
wazuh-indexer_1 | [2022-01-26T19:50:47,858][INFO ][o.o.p.PluginsService ] [wazuh-indexer] loaded plugin [opensearch-security]
wazuh-indexer_1 | [2022-01-26T19:50:47,859][INFO ][o.o.p.PluginsService ] [wazuh-indexer] loaded plugin [opensearch-sql]
wazuh-indexer_1 | [2022-01-26T19:50:47,889][INFO ][o.o.s.OpenSearchSecurityPlugin] [wazuh-indexer] Disabled https compression by default to mitigate BREACH attacks. You can enable it by setting 'http.compression: true' in opensearch.yml
wazuh-indexer_1 | [2022-01-26T19:50:47,961][INFO ][ooeNodeEnvironment ] [wazuh-indexer] using [1] data paths, mounts [[/var/lib/wazuh-indexer (/dev/sda5)]], net usable_space [58.7gb], net total_space [100gb], types [ext4]
wazuh-indexer_1 | [2022-01-26T19:50:47,990][INFO ][o.o.e.NodeEnvironment ] [wazuh-indexer] heap size [1gb], compressed ordinary object pointers [true]
wazuh-indexer_1 | [2022-01-26T19:50:48,151][INFO ][oonNode ] [wazuh-indexer] node name [wazuh-indexer], node ID [mLVp-mfJTVKNtyCp1vfb-w], cluster name [wazuh-cluster], roles [master, remote_cluster_client, data, ingest]

The certificate creation yaml was used, but the default configuration threw the first error:

$ docker-compose -f generate-indexer-certs.yml run --rm generator
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "node_type" (class com.floragunn.searchguard.tools.tlstool.Config$Node), not marked as ignorant (9 known properties: "keysize", "pkPassword" , "ip", "dns", "dn", "name", "ellipticCurve", "validityDays", "oid"])
 at [Source: (File); line: 15, column: 24] (through reference chain: com.floragunn.searchguard.tools.tlstool.Config["nodes"]->java.util.ArrayList[3]->com.floragunn.searchguard.tools.tlstool .Config$Node["node_type"])
No files have been written
Success! Exiting.

Parameters that were generating errors were eliminated, but it continues with other errors:

$ docker-compose -f generate-indexer-certs.yml run --rm generator
Configuration ca is required
No files have been written
Success! Exiting.

The investigation continues.

vcerenu commented 2 years ago

The opensearch.yml files corresponding to each one of the nodes were revised, the dn were modified and the certs.yml file used as a template for the creation of the keys and certificates of the wazuh-indexer nodes was modified.

It was found that the certificates and keys require specific permissions and that the wazuh-indexer user be the owner of the files, so the creation file was modified to generate them with these permissions:

chown 998:996 *
chmod 0600 *

Cluster startup tests were carried out, it usually presents problems when it is assembled since some nodes fail to join the cluster or have unstable behavior:

vcerenu@vcerenu-VirtualBox:~/Repositories/wazuh-docker$ curl -k -u admin:admin https://127.0.0.1:9700/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
172.29.0.6 56 98 6 4.78 2.91 1.95 dimr * wazuh-indexer
172.29.0.2 17 98 6 4.78 2.91 1.95 dimr -wazuh-indexer-3
172.29.0.4 43 98 6 4.78 2.91 1.95 dimr - wazuh-indexer-2
vcerenu@vcerenu-VirtualBox:~/Repositories/wazuh-docker$ curl -k -u admin:admin https://127.0.0.1:9700/_cat/nodes?v
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 127.0.0.1:9700
vcerenu@vcerenu-VirtualBox:~/Repositories/wazuh-docker$ curl -k -u admin:admin https://127.0.0.1:9700/_cat/nodes?v
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 127.0.0.1:9700
vcerenu@vcerenu-VirtualBox:~/Repositories/wazuh-docker$ curl -k -u admin:admin https://127.0.0.1:9700/_cat/nodes?v
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 127.0.0.1:9700
vcerenu@vcerenu-VirtualBox:~/Repositories/wazuh-docker$ curl -k -u admin:admin https://127.0.0.1:9700/_cat/nodes?v
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 127.0.0.1:9700
vcerenu@vcerenu-VirtualBox:~/Repositories/wazuh-docker$ curl -k -u admin:admin https://127.0.0.1:9700/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
172.29.0.2 63 62 21 1.19 1.97 2.01 dimr * node-1
vcerenu@vcerenu-VirtualBox:~/Repositories/wazuh-docker$ curl -k -u admin:admin https://127.0.0.1:9700/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.0.5 9 98 12 2.18 2.33 1.44 dimr * wazuh-indexer
vcerenu@vcerenu-VirtualBox:~/Repositories/wazuh-docker$ curl -k -u admin:admin https://127.0.0.1:9700/_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.0.2 38 98 24 1.64 2.09 1.41 dimr-wazuh-indexer-3
192.168.0.5 16 98 25 1.64 2.09 1.41 dimr * wazuh-indexer
192.168.0.3 31 98 32 1.64 2.09 1.41 dimr-wazuh-indexer-2

It is being tested if at the moment of starting the master node it needs a certain time to be able to start receiving the other nodes and if they require restarts to be able to stabilize after starting.

vcerenu commented 2 years ago

Made changes to the wazuh-indexer entrypoint so that the boot times of the node where securityadmin.sh is run has time to successfully execute the command and then the child nodes start.

It is possible to raise the cluster after changing the execution of wazuh-indexer from the binary to the service, since when verifying between the nodes they obtained that the service was down and did not respond. This change made the nodes able to see each other and initialize the cluster, but wazuh-indexer cannot be accessed on port 9700 as it gives the following error:

$ curl -k -u admin:admin https://127.0.0.1:9700/_cat/nodes?v
unauthorized
$

Investigation of this bug continues.

Wazuh-dashboard does not reach the cluster and sends the following error message:

{"type":"log","@timestamp":"2022-01-28T20:04:52Z","tags":["error","opensearch","data"],"pid":11, "message":"[ConnectionError]: unable to get local issuer certificate"}

In the log of wazuh-manager it is seen that it reaches the cluster, but it logs the following error:

2022-01-28T20:04:21.194Z WARN [elasticsearch] elasticsearch/client.go:408 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xc07530de96f60eec, ext:12065819825, loc: (*time.Location)(0x42417a0)}, Meta:{"pipeline":"filebeat-7.10.2-wazuh-alerts-pipeline"}, Fields:{"agent":{"ephemeral_id":"c34cfad5-0dec- 4545-af04-f2e1d7c690c1","hostname":"wazuh-worker","id":"c198240c-b245-4d7b-a4c9-2e8f5e871a87","name":"wazuh-worker","type":"filebeat" ,"version":"7.10.2"},"ecs":{"version":"1.6.0"},"event":{"dataset":"wazuh.alerts","module":"wazuh" },"fields":{"index_prefix":"wazuh-alerts-4.x-"},"fileset":{"name":"alerts"},"host":{"name":"wazuh-worker "},"input":{"type":"log"},"log":{"file":{"path":"/var/ossec/logs/alerts/alerts.json"},"offset" :330805},"message":"{\"timestamp\":\"2022-01-28T20:03:04.999+0000\",\"rule\":{\"level\":7,\"description \":\"CIS Benchmark for CentOS 7: Ensure shadow group is empty\",\"id\":\"19007\",\"firedtimes\":83,\"mail\":false,\"groups \":[\"sca\"],\"gd pr\":[\"IV_35.7.d\"],\"pci_dss\":[\"2.2\",\"2.2.3\"],\"nist_800_53\":[\"CM.1 \",\"CM.1\"],\"tsc\":[\"CC7.1\",\"CC7.2\",\"CC5.2\"],\"cis\": [\"6.2.18\"],\"cis_csc\":[\"5.1\"],\"gpg_13\":[\"4.3\"],\"gdpr_IV\":[\"35.7.d \"],\"hipaa\":[\"164.312.b\"]},\"agent\":{\"id\":\"000\",\"name\":\"wazuh- worker\"},\"manager\":{\"name\":\"wazuh-worker\"},\"id\":\"1643400184.545808\",\"cluster\":{\"name\ ":\"wazuh\",\"node\":\"worker01\"},\"decoder\":{\"name\":\"sca\"},\"data\":{\" sca\":{\"type\":\"check\",\"scan_id\":\"267118688\",\"policy\":\"CIS Benchmark for CentOS 7\",\"check\" :{\"id\":\"6195\",\"title\":\"Ensure shadow group is empty\",\"description\":\"The shadow group allows system programs which require access the ability to read the /etc/shadow file. No users should be assigned to the shadow group\",\"rationale\":\"Any users assigned to the shadow group would be granted read access to the /etc/shadow file. If attackers can gain read access to the /etc/shadow file, they can easily run a password cracking program against the hashed passwords to break them. Other security information that is stored in the /etc/shadow file (such as expiration) could also be useful to subvert additional user accounts.\",\"remediation\":\"Remove any legacy '+' entries from /etc/ shadow if they exist.\",\"compliance\":{\"cis\":\"6.2.18\",\"cis_csc\":\"5.1\",\"pci_dss\":\"2.2 .3\",\"nist_800_53\":\"CM.1\",\"gpg_13\":\"4.3\",\"gdpr_IV\":\"35.7.d\",\"hipaa\" :\"164.312.b\",\"tsc\":\"CC5.2\"},\"file\":[\"/etc/shadow\"],\"result\":\"failed \"}}},\"location\":\"sca\"}","service":{"type":"wazuh"}}, Private:file.State{Id:"native::4588016-2053 ", PrevId:"", Finished:false, Fileinfo:(*os.fileStat)(0xc00067de10), Source:"/var/ossec/logs/alerts/alerts.json", Offset:332388, Timestamp:time.Time{ wall:0xc07530db9681a0c3, ext:58189428, loc:(*time.Location)(0x42417a0)}, TTL:-1, Type:"log", Meta:map[string]string(nil), FileStateOS:file.StateOS{Inode :0x4601f0, Device:0x805}, IdentifierName:"native"}, TimeSeries:false}, Flags:0x1, Cache:publisher.EventCache{m:common.MapStr(nil)}} (sta your=400): {"type":"illegal_argument_exception","reason":"mapper [decoder.name] cannot be changed from type [keyword] to [text]"}

The investigation of the reason for this error continues, since it seems that the indexes are not ready to receive the data in the format that is sent.

vcerenu commented 2 years ago

A new Dockerfile was created for the build of the wazuh-indexer image from the base used in the creation of the RPM package, it is developed in the file wazuh-indexer/Dockerfile_new and its entrypoint is developed in the file wazuh- indexer/config/config.sh.

The script wazuh-indexer/config/tarball.sh was created for the creation of a new base that is more friendly at the time of the construction of the Docker image of wazuh-indexer, work continues on this procedure.

The build of the Docker image for the creation of the wazuh-indexer cluster deployment certificates was modified, since the previously created certificates generate errors when running the securityadmin.sh script, it is still under development.

vcerenu commented 2 years ago

A new Dockerfile was created which performs a multistage build, assembling the entire directory structure of the wazuh-indexer application and then moving all this to a new image. In addition, the entrypoint of the image was modified so that it takes these changes.

Changed the opensearch.yml files according to the opensearch documentation, since the main node does not run the securityadmin.sh script. This is a example of one of the nodes:

network.host: wazuh-indexer
node.name: wazuh-indexer
cluster.initial_master_nodes:
        - wazuh-indexer
        - wazuh-indexer-2
        - wazuh-indexer-3
cluster.name: "wazuh-cluster"
discovery.seed_hosts:
        - wazuh-indexer
        - wazuh-indexer-2
        - wazuh-indexer-3
http.port: 9700-9799
transport.tcp.port: 9800-9899
node.max_local_storage_nodes: "3"
path.data: /var/lib/wazuh-indexer
path.logs: /var/log/wazuh-indexer
###############################################################################
#                                                                             #
#         WARNING: Insecure demo certificates set up in this file.            #
#                  Please change on production cluster!                       #
#                                                                             #
###############################################################################
plugins.security.ssl.http.pemcert_filepath: /etc/wazuh-indexer/certs/wazuh-indexer.pem
plugins.security.ssl.http.pemkey_filepath: /etc/wazuh-indexer/certs/wazuh-indexer.key
plugins.security.ssl.http.pemtrustedcas_filepath: /etc/wazuh-indexer/certs/root-ca.pem
plugins.security.ssl.transport.pemcert_filepath: /etc/wazuh-indexer/certs/wazuh-indexer.pem
plugins.security.ssl.transport.pemkey_filepath: /etc/wazuh-indexer/certs/wazuh-indexer.key
plugins.security.ssl.transport.pemtrustedcas_filepath: /etc/wazuh-indexer/certs/root-ca.pem
plugins.security.ssl.http.enabled: true
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.transport.resolve_hostname: false
plugins.security.audit.type: internal_opensearch
plugins.security.authcz.admin_dn:
- "CN=admin,OU=Docu,O=Wazuh,L=California,C=US"
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.nodes_dn:
- "CN=wazuh-indexer,OU=Docu,O=Wazuh,L=California,C=US"
- "CN=wazuh-indexer-2,OU=Docu,O=Wazuh,L=California,C=US"
- "CN=wazuh-indexer-3,OU=Docu,O=Wazuh,L=California,C=US"
- "CN=filebeat,OU=Docu,O=Wazuh,L=California,C=US"
plugins.security.restapi.roles_enabled:
- "all_access"
- "security_rest_api_access"
plugins.security.allow_default_init_securityindex: true
cluster.routing.allocation.disk.threshold_enabled: false
opendistro_security.audit.config.disabled_rest_categories: NONE
opendistro_security.audit.config.disabled_transport_categories: NONE

A startup test of a 3-node cluster was performed and the test worked, now it only remains to check wazuh-dashboard, which fails to connect to the cluster, logging the following error:

{"type":"log","@timestamp":"2022-02-04T20:44:24Z","tags":["error","opensearch","data"],"pid":11,"message":"[ConnectionError]: unable to get local issuer certificate"}
vcerenu commented 2 years ago

All service names have been revised to match the names of the applications they run.

A new yaml script was created for the execution of the stack formed by wazuh-manager, wazuh-indexer and wazuh-dashboard, they have reforms on the received configuration files (certificates and opensearch.yml) so that their nomenclature is more descriptive. Example of opensearch.yml:

network.host: wazuh1-indexer
node.name: wazuh1-indexer
cluster.initial_master_nodes:
        - wazuh1-indexer
        - wazuh2-indexer
        - wazuh3-indexer
cluster.name: "wazuh-cluster"
discovery.seed_hosts:
        - wazuh1-indexer
        - wazuh2-indexer
        - wazuh3-indexer
http.port: 9700-9799
transport.tcp.port: 9800-9899
node.max_local_storage_nodes: "3"
path.data: /var/lib/wazuh-indexer
path.logs: /var/log/wazuh-indexer
###############################################################################
#                                                                             #
#         WARNING: Insecure demo certificates set up in this file.            #
#                  Please change on production cluster!                       #
#                                                                             #
###############################################################################
plugins.security.ssl.http.pemcert_filepath: /etc/wazuh-indexer/certs/wazuh1-indexer.pem
plugins.security.ssl.http.pemkey_filepath: /etc/wazuh-indexer/certs/wazuh1-indexer.key
plugins.security.ssl.http.pemtrustedcas_filepath: /etc/wazuh-indexer/certs/root-ca.pem
plugins.security.ssl.transport.pemcert_filepath: /etc/wazuh-indexer/certs/wazuh1-indexer.pem
plugins.security.ssl.transport.pemkey_filepath: /etc/wazuh-indexer/certs/wazuh1-indexer.key
plugins.security.ssl.transport.pemtrustedcas_filepath: /etc/wazuh-indexer/certs/root-ca.pem
plugins.security.ssl.http.enabled: true
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.transport.resolve_hostname: false
plugins.security.audit.type: internal_opensearch
plugins.security.authcz.admin_dn:
- "CN=admin,OU=Docu,O=Wazuh,L=California,C=US"
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.nodes_dn:
- "CN=wazuh1-indexer,OU=Docu,O=Wazuh,L=California,C=US"
- "CN=wazuh2-indexer,OU=Docu,O=Wazuh,L=California,C=US"
- "CN=wazuh3-indexer,OU=Docu,O=Wazuh,L=California,C=US"
- "CN=filebeat,OU=Docu,O=Wazuh,L=California,C=US"
plugins.security.restapi.roles_enabled:
- "all_access"
- "security_rest_api_access"
plugins.security.allow_default_init_securityindex: true
cluster.routing.allocation.disk.threshold_enabled: false
opendistro_security.audit.config.disabled_rest_categories: NONE
opendistro_security.audit.config.disabled_transport_categories: NONE

A new image was created, which automatically generates the certificates for Wazuh manager, the Wazuh indexer cluster and Wazuh dashboard, leaving them available so that the Docker Compose file that we execute when initializing the stack has them available.. The execution of this image is parameterized in the generate-indexer-certs.yml script:

# Wazuh App Copyright (C) 2021 Wazuh Inc. (License GPLv2)
version: '3'

services:
  generator:
    image: wazuh/wazuh-certs-generator:0.0.1
    hostname: wazuh-certs-generator
    volumes:
      - ./production_cluster/wazuh_indexer_ssl_certs/certs.yml:/unattended_installer/install_functions/config.yml
      - ./production_cluster/wazuh_indexer_ssl_certs/:/unattended_installer/install_functions/certificates/
vcerenu commented 2 years ago

The stack assembly was being tested within the test_cluster.yml script, which I have found some problems:

An error was found in the wazuh-manager cluster startup, which is assigned to the corresponding team in issue 574.

Because version 4.3.0 rc3 had the error at startup, we tried to start the cluster with the 4.3.0 rc1 image, but apparently there are some differences in the templates:

wazuh3.indexer_1   | [2022-02-08T20:20:22,398][INFO ][o.o.a.b.TransportShardBulkAction] [wazuh3.indexer] [wazuh-alerts-4.x-2022.02.08][0] mapping update rejected by primary
wazuh3.indexer_1   | java.lang.IllegalArgumentException: mapper [decoder.name] cannot be changed from type [keyword] to [text]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.ParametrizedFieldMapper.merge(ParametrizedFieldMapper.java:110) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.ParametrizedFieldMapper.merge(ParametrizedFieldMapper.java:74) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.ObjectMapper.doMerge(ObjectMapper.java:585) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.ObjectMapper.merge(ObjectMapper.java:544) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.ObjectMapper.doMerge(ObjectMapper.java:571) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.RootObjectMapper.doMerge(RootObjectMapper.java:330) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.ObjectMapper.merge(ObjectMapper.java:544) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.RootObjectMapper.merge(RootObjectMapper.java:325) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.Mapping.merge(Mapping.java:127) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.DocumentMapper.merge(DocumentMapper.java:306) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.MapperService.internalMerge(MapperService.java:567) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.MapperService.internalMerge(MapperService.java:512) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.index.mapper.MapperService.merge(MapperService.java:453) ~[opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:372) [opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.action.bulk.TransportShardBulkAction$2.doRun(TransportShardBulkAction.java:212) [opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:50) [opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.action.bulk.TransportShardBulkAction.performOnPrimary(TransportShardBulkAction.java:276) [opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.action.bulk.TransportShardBulkAction.dispatchedShardOperationOnPrimary(TransportShardBulkAction.java:165) [opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.action.bulk.TransportShardBulkAction.dispatchedShardOperationOnPrimary(TransportShardBulkAction.java:98) [opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.action.support.replication.TransportWriteAction$1.doRun(TransportWriteAction.java:220) [opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:792) [opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:50) [opensearch-1.2.3.jar:1.2.3]
wazuh3.indexer_1   |    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
wazuh3.indexer_1   |    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
wazuh3.indexer_1   |    at java.lang.Thread.run(Thread.java:832) [?:?]

Check connection between wazuh-dashboard and wazuh-indexer. The indexer cluster starts correctly but from the dashboard I get the following error:

wazuh.dashboard_1  | {"type":"log","@timestamp":"2022-02-08T19:20:37Z","tags":["error","opensearch","data"],"pid":11,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh1-indexer wazuh1-indexer:9700"}
wazuh.dashboard_1  | {"type":"log","@timestamp":"2022-02-08T19:20:40Z","tags":["error","opensearch","data"],"pid":11,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh1-indexer wazuh1-indexer:9700"}
wazuh.dashboard_1  | {"type":"log","@timestamp":"2022-02-08T19:20:42Z","tags":["error","opensearch","data"],"pid":11,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh1-indexer wazuh1-indexer:9700"}
wazuh.dashboard_1  | {"type":"log","@timestamp":"2022-02-08T19:20:45Z","tags":["error","opensearch","data"],"pid":11,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh1-indexer wazuh1-indexer:9700"}
wazuh.dashboard_1  | {"type":"log","@timestamp":"2022-02-08T19:20:47Z","tags":["error","opensearch","data"],"pid":11,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh1-indexer wazuh1-indexer:9700"}
wazuh.dashboard_1  | {"type":"log","@timestamp":"2022-02-08T19:20:50Z","tags":["error","opensearch","data"],"pid":11,"message":"[ConnectionError]: getaddrinfo ENOTFOUND wazuh1-indexer wazuh1-indexer:9700"}

The connection from the dashboard container to indexer was checked and it works correctly, so we continue to check its SSL configuration:

uid=0(root) gid=0(root) groups=0(root)
root@wazuh:/# uname -a
Linux wazuh.dashboard 5.13.0-27-generic #29~20.04.1-Ubuntu SMP Fri Jan 14 00:32:30 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
root@wazuh:/# curl -k -u admin:SecretPassword https://wazuh1.indexer:9700/_cat/nodes?v
ip         heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
172.28.0.5           45          98  25    2.44    1.13     0.47 dimr      -      wazuh2.indexer
172.28.0.3           25          98  25    2.44    1.13     0.47 dimr      *      wazuh1.indexer
172.28.0.4           41          98  26    2.44    1.13     0.47 dimr      -      wazuh3.indexer
root@wazuh:/#
vcerenu commented 2 years ago

Check connection of wazuh-dashboard against wazuh-indexer. The dashboard.yml file was verified to have settings that were not compatible with the Docker stack deployment

A dashboard.yml file was mounted in the wazuh.dashboard container of the stack to be able to configure the wazuh.dashboard container with all the options.

server.host: 0.0.0.0
server.port: 5601
opensearch.hosts: https://wazuh1.indexer:9700
opensearch.ssl.verificationMode: certificate
opensearch.username: kibanaserver
opensearch.password: kibanaserver
opensearch.requestHeadersWhitelist: ["securitytenant","Authorization"]
opensearch_security.multitenancy.enabled: false
opensearch_security.readonly_mode.roles: ["kibana_read_only"]
server.ssl.enabled: true
server.ssl.key: "/etc/wazuh-dashboard/certs/wazuh-dashboard-key.pem"
server.ssl.certificate: "/etc/wazuh-dashboard/certs/wazuh-dashboard.pem"
opensearch.ssl.certificateAuthorities: ["/etc/wazuh-dashboard/certs/root-ca.pem"]
uiSettings.overrides.defaultRoute: /app/wazuh?security_tenant=global

It was verified that wazuh was accessible at the url https://0.0.0.0:443

It was verified that there was no connection against the wazuh API. the wazuh.yml file does not have the necessary configuration. A solution is being worked on to modify these parameters dynamically

vcerenu commented 2 years ago

Tests were performed to modify the data in the wazuh.yml file that parameterizes wazuh-dashboard's access to wazuh-manager via the API. When trying to modify this file within the start of the container, we had errors because this parameterization is done after the start, so the file was attached by means of a volume, giving the necessary permissions on the directories that were created for its location.

With these changes made we were able to initialize wazuh-indexer, wazuh-dashboard and connect to the app

image

image

image

We are verifying the start of the test stack but we have errors in the start of wazuh-indexer in single-node mode When starting this stack we get the following errors:

wazuh1.indexer_1  | [2022-02-10T19:54:49,724][ERROR][o.o.s.a.BackendRegistry  ] [wazuh1.indexer] Not yet initialized (you may need to run securityadmin)
wazuh1.indexer_1  | [2022-02-10T19:54:50,461][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [wazuh1.indexer] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)
wazuh1.indexer_1  | [2022-02-10T19:54:50,462][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [wazuh1.indexer] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)
wazuh1.indexer_1  | [2022-02-10T19:54:50,462][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [wazuh1.indexer] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)
wazuh1.indexer_1  | [2022-02-10T19:54:50,462][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [wazuh1.indexer] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)
wazuh1.indexer_1  | [2022-02-10T19:54:50,462][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [wazuh1.indexer] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)
wazuh1.indexer_1  | [2022-02-10T19:54:50,463][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [wazuh1.indexer] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)
wazuh1.indexer_1  | [2022-02-10T19:54:50,463][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [wazuh1.indexer] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)
wazuh1.indexer_1  | [2022-02-10T19:54:50,463][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [wazuh1.indexer] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)
wazuh1.indexer_1  | [2022-02-10T19:54:50,463][ERROR][o.o.s.c.ConfigurationLoaderSecurity7] [wazuh1.indexer] Failure no such index [.opendistro_security] retrieving configuration for [INTERNALUSERS, ACTIONGROUPS, CONFIG, ROLES, ROLESMAPPING, TENANTS, NODESDN, WHITELIST, AUDIT] (index=.opendistro_security)
wazuh1.indexer_1  | [2022-02-10T19:54:54,337][INFO ][o.o.i.i.ManagedIndexCoordinator] [wazuh1.indexer] Performing move cluster state metadata.

These errors show that the securityadmin.sh script is not running correctly.

When checking inside the container we did the test of running securityadmin.sh by hand and we had errors if we used localhost or 127.0.0.1 as hostname, but when changing to the name of the container it works:

root@wazuh1:/etc/wazuh-indexer/certs# export JAVA_HOME=/usr/share/wazuh-indexer/jdk/ &&  bash /usr/share/wazuh-indexer/plugins/opensearch-security/tools/securityadmin.sh -cd /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/ -nhnv -cacert /etc/wazuh-indexer/certs/root-ca.pem -cert /etc/wazuh-indexer/certs/admin.pem -key /etc/wazuh-indexer/certs/admin-key.pem -p 9800 -icl
Security Admin v7
Will connect to localhost:9800
ERR: Seems there is no OpenSearch running on localhost:9800 - Will exit
root@wazuh1:/etc/wazuh-indexer/certs# export JAVA_HOME=/usr/share/wazuh-indexer/jdk/ &&  bash /usr/share/wazuh-indexer/plugins/opensearch-security/tools/securityadmin.sh -cd /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/ -nhnv -cacert /etc/wazuh-indexer/certs/root-ca.pem -cert /etc/wazuh-indexer/certs/admin.pem -key /etc/wazuh-indexer/certs/admin-key.pem -p 9800 -icl -h 127.0.0.1
Security Admin v7
Will connect to 127.0.0.1:9800
ERR: Seems there is no OpenSearch running on 127.0.0.1:9800 - Will exit
root@wazuh1:/etc/wazuh-indexer/certs# export JAVA_HOME=/usr/share/wazuh-indexer/jdk/ &&  bash /usr/share/wazuh-indexer/plugins/opensearch-security/tools/securityadmin.sh -cd /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/ -nhnv -cacert /etc/wazuh-indexer/certs/root-ca.pem -cert /etc/wazuh-indexer/certs/admin.pem -key /etc/wazuh-indexer/certs/admin-key.pem -p 9800 -icl -h wazuh1.indexer
Security Admin v7
Will connect to wazuh1.indexer:9800 ... done
Connected as CN=admin,OU=Demo,O=Wazuh,L=California,C=US
OpenSearch Version: 1.2.3
OpenSearch Security Version: 1.2.3.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: opensearch
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index does not exists, attempt to create it ... done (0-all replicas)
Populate config from /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/
Will update '_doc/config' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/config.yml 
   SUCC: Configuration for 'config' created or updated
Will update '_doc/roles' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/roles.yml 
   SUCC: Configuration for 'roles' created or updated
Will update '_doc/rolesmapping' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/roles_mapping.yml 
   SUCC: Configuration for 'rolesmapping' created or updated
Will update '_doc/internalusers' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/internal_users.yml 
   SUCC: Configuration for 'internalusers' created or updated
Will update '_doc/actiongroups' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/action_groups.yml 
   SUCC: Configuration for 'actiongroups' created or updated
Will update '_doc/tenants' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/tenants.yml 
   SUCC: Configuration for 'tenants' created or updated
Will update '_doc/nodesdn' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/nodes_dn.yml 
   SUCC: Configuration for 'nodesdn' created or updated
Will update '_doc/whitelist' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/whitelist.yml 
   SUCC: Configuration for 'whitelist' created or updated
Will update '_doc/audit' with /usr/share/wazuh-indexer/plugins/opensearch-security/securityconfig/audit.yml 
   SUCC: Configuration for 'audit' created or updated
Done with success
root@wazuh1:/etc/wazuh-indexer/certs#

We continue with the tests of this environment

vcerenu commented 2 years ago

A check was added for the execution of the securityadmin.sh script, checking if the parameter discovery.type has the value single-node, since it did not execute the script when this type of wazuh-indexer instance was created.

export DISCOVERY=$(grep -oP "(?<=discovery.type: ).*" /etc/wazuh-indexer/opensearch.yml)
export CACERT=$(grep -oP "(?<=plugins.security.ssl.transport.pemtrustedcas_filepath: ).*" /etc/wazuh-indexer/opensearch.yml)
export CERT="/etc/wazuh-indexer/certs/admin.pem"
export KEY="/etc/wazuh-indexer/certs/admin-key.pem"
...

if [[ "$DISCOVERY" == "single-node" ]]; then
   # run securityadmin.sh for single node
   nohup /securityadmin.sh &
fi

A new configuration was added to the wazuh-dashboard dashboard.yml file since when the test environment was built, the file that came with the package did not start

vcerenu commented 2 years ago

Problems were found when initializing the Wazuhstack, specifically with wazuh-dashboard. When starting the application, it generates the directories where it adds the connection data with the wazuh-manager API, this means that changes cannot be made before the application starts.

At first we tried to mount the configuration file but we got errors creating the directories where the configuration and log files were later housed.

It was found as an alternative to create these directories in advance, in the Dockerfile itself, assign the necessary permissions so that the subsequent creation attempt does not generate errors and has the directory scheme that it needs.

Test stack deployment and production stack were tested and both started successfully.

It is required to modify the documentation to change the steps that correspond to the creation of certificates for wazuh-indexer and wazuh-dashboard, which were simplified into a single step and an image was created that automatically generates these certificates.

vcerenu commented 2 years ago

Created a new yaml script for building images of wazuh-manager, wazuh-indexer, wazuh-dashboard and the certificates creator.

The files that corresponded to the build of images and deploy of the Wazuh stack with odfe were eliminated.

Directories that are being used for creating images of the new stack were renamed so they are used in the new procedure.

This is the new directory structure, which contemplates the creation of images of wazuh products and the deployment of these products and xpack:

├── build-wazuh-images.yml
├── CHANGELOG.md
├── docker-compose.yml
├── generate-elasticsearch-certs.yml
├── generate-indexer-certs.yml
├── indexer_certs_creator
│   ├── config
│   │   ├── entrypoint.sh
│   │   └── unattended_installer.tar.gz
│   └── Dockerfile
├── kibana
│   ├── config
│   │   ├── entrypoint.sh
│   │   ├── kibana_settings.sh
│   │   ├── wazuh_app_config.sh
│   │   ├── wazuh.yml
│   │   └── xpack_config.sh
│   └── Dockerfile
├── LICENSE
├── production_cluster
│   ├── nginx
│   │   ├── nginx.conf
│   │   └── ssl
│   │       └── generate-self-signed-cert.sh
│   ├── wazuh_cluster
│   │   ├── wazuh_manager.conf
│   │   └── wazuh_worker.conf
│   ├── wazuh_dashboard
│   │   └── dashboard.yml
│   ├── wazuh-indexer
│   │   ├── internal_users.yml
│   │   ├── opensearch.yml
│   │   ├── wazuh1.indexer.yml
│   │   ├── wazuh2.indexer.yml
│   │   └── wazuh3.indexer.yml
│   └── wazuh_indexer_ssl_certs
│       └── certs.yml
├── production-cluster.yml
├── README.md
├── VERSION
├── wazuh-dashboard
│   ├── config
│   │   ├── dashboard.yml
│   │   ├── entrypoint.sh
│   │   ├── wazuh_app_config.sh
│   │   └── wazuh.yml
│   └── Dockerfile
├── wazuh-indexer
│   ├── config
│   │   ├── config.sh
│   │   ├── entrypoint.sh
│   │   ├── opensearch.yml
│   │   ├── securityadmin.sh
│   │   └── unattended_installer.tar.gz
│   └── Dockerfile
├── wazuh-manager
│   ├── config
│   │   ├── create_user.py
│   │   ├── etc
│   │   │   ├── cont-init.d
│   │   │   │   ├── 0-wazuh-init
│   │   │   │   ├── 1-config-filebeat
│   │   │   │   └── 2-manager
│   │   │   └── services.d
│   │   │       ├── filebeat
│   │   │       │   ├── finish
│   │   │       │   └── run
│   │   │       └── ossec-logs
│   │   │           └── run
│   │   ├── filebeat.yml
│   │   ├── permanent_data.env
│   │   ├── permanent_data.sh
│   │   └── wazuh.repo
│   └── Dockerfile
├── xpack
│   └── instances.yml
├── xpack-compose.yml
└── xpack-from-sources.yml
vcerenu commented 2 years ago

Updated wazuh-indexer base package, which does not include demo certificates.

A procedure was created to create demo certificates forwazuh-indexer and that they remain in the same directory where the previous certificates were stored, which was resolved in the builder image so as not to add cache to the final image of wazuh-indexer.

Several corrections were also made to the PR https://github.com/wazuh/wazuh-docker/pull/577 requested in a review.

teddytpc1 commented 2 years ago

Some changes related to the way of downloading the cert and password tools are being made.

teddytpc1 commented 2 years ago

The tools download logic was updated to download the script from packages o packages-dev depending on wether the files exist in the buckets and returning an exit 1 if the do not exist in either repositories. The PR was created: #584

teddytpc1 commented 2 years ago

The username and workdir was updated for Indexer and Dashboard images. The PR was updated:

577

This was tested locally with the packages-dev repository.

teddytpc1 commented 2 years ago

In the Wazuh dashboard docker image, some configurations were in /etc/wazuh-dashboard. These files should be under /usr/share/wazuh-dashboard/config. Some tests to change it were made. In the process we tried to modify the image creation in order to start using the Wazuh dashboard base zip file instead of the package. It is not fully working yet.

teddytpc1 commented 2 years ago

The wazuh-dashboard image creation has been updated and it is working. Changes were pushed to the new-packages-release-dashboard branch. A PR will be created.