wazuh / wazuh-packages

Wazuh - Tools for packages creation
https://wazuh.com
GNU General Public License v2.0
105 stars 98 forks source link

Missing files in the indexer by uninstalling, cleaning indexer directories and reinstalling it #3032

Closed santipadilla closed 4 months ago

santipadilla commented 4 months ago

Description

opensearch.yml file in 4.8.0 ```console root@wazuh-indexer-1:/home/vagrant# ls /etc/wazuh-indexer/ certs opensearch.keystore opensearch-performance-analyzer jvm.options opensearch-notifications opensearch-reports-scheduler jvm.options.d opensearch-notifications-core opensearch-security log4j2.properties opensearch-observability opensearch.yml ```
remove opensearch.yml ```console root@wazuh-indexer-1:/home/vagrant# rm /etc/wazuh-indexer/opensearch.yml root@wazuh-indexer-1:/home/vagrant# ls /etc/wazuh-indexer/ certs opensearch.keystore opensearch-performance-analyzer jvm.options opensearch-notifications opensearch-reports-scheduler jvm.options.d opensearch-notifications-core opensearch-security log4j2.properties opensearch-observability ```
uninstall without purge ```console root@wazuh-indexer-1:/home/vagrant# apt-get remove wazuh-indexer -y Reading package lists... Done Building dependency tree... Done Reading state information... Done The following packages will be REMOVED: wazuh-indexer 0 upgraded, 0 newly installed, 1 to remove and 164 not upgraded. After this operation, 1,050 MB disk space will be freed. (Reading database ... 77203 files and directories currently installed.) Removing wazuh-indexer (4.8.0-1) ... Stopping wazuh-indexer service... OK ```
Install v4.8.1 and opensearch.yml file is missing ```console root@wazuh-indexer-1:/home/vagrant# ls /etc/wazuh-indexer/ certs opensearch.keystore opensearch-performance-analyzer jvm.options opensearch-notifications opensearch-reports-scheduler jvm.options.d opensearch-notifications-core opensearch-security log4j2.properties opensearch-observability ```
AlexRuiz7 commented 4 months ago

I think this is expected. I haven't tested it, but if the package detects a previous installation (if /etc/wazuh-indexer folder exists), it doesn't overwrite the configuration files. We will need to test this behavior in OpenSearch to be sure.

santipadilla commented 4 months ago
rauldpm commented 4 months ago

Update report

install 4.8.0 -> install 4.8.1 :green_circle: ``` root@ubuntu22:/home/vagrant# apt install ./wazuh-indexer_4.8.0-1_amd64.deb Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'wazuh-indexer' instead of './wazuh-indexer_4.8.0-1_amd64.deb' The following NEW packages will be installed: wazuh-indexer 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/759 MB of archives. After this operation, 1,050 MB of additional disk space will be used. Get:1 /home/vagrant/wazuh-indexer_4.8.0-1_amd64.deb wazuh-indexer amd64 4.8.0-1 [759 MB] Selecting previously unselected package wazuh-indexer. (Reading database ... 76324 files and directories currently installed.) Preparing to unpack .../wazuh-indexer_4.8.0-1_amd64.deb ... Creating wazuh-indexer group... OK Creating wazuh-indexer user... OK Unpacking wazuh-indexer (4.8.0-1) ... Setting up wazuh-indexer (4.8.0-1) ... Created opensearch keystore in /etc/wazuh-indexer/opensearch.keystore Processing triggers for libc-bin (2.35-0ubuntu3.6) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. N: Download is performed unsandboxed as root as file '/home/vagrant/wazuh-indexer_4.8.0-1_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) root@ubuntu22:/home/vagrant# ls -l /etc/wazuh-indexer/ total 56 -rw-rw---- 1 wazuh-indexer wazuh-indexer 2937 Jul 10 12:55 jvm.options drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jun 6 12:10 jvm.options.d -rw-r----- 1 wazuh-indexer wazuh-indexer 14808 Jun 6 12:10 log4j2.properties -rw-rw---- 1 wazuh-indexer wazuh-indexer 196 Jul 10 12:55 opensearch.keystore drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:55 opensearch-notifications drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:55 opensearch-notifications-core drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:55 opensearch-observability drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:55 opensearch-performance-analyzer drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:55 opensearch-reports-scheduler drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:55 opensearch-security -rw-rw---- 1 wazuh-indexer wazuh-indexer 2081 Jun 6 12:10 opensearch.yml root@ubuntu22:/home/vagrant# apt install ./wazuh-indexer_4.8.1-1_amd64.deb Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'wazuh-indexer' instead of './wazuh-indexer_4.8.1-1_amd64.deb' The following packages will be upgraded: wazuh-indexer 1 upgraded, 0 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/759 MB of archives. After this operation, 0 B of additional disk space will be used. Get:1 /home/vagrant/wazuh-indexer_4.8.1-1_amd64.deb wazuh-indexer amd64 4.8.1-1 [759 MB] (Reading database ... 77497 files and directories currently installed.) Preparing to unpack .../wazuh-indexer_4.8.1-1_amd64.deb ... Unpacking wazuh-indexer (4.8.1-1) over (4.8.0-1) ... Setting up wazuh-indexer (4.8.1-1) ... Processing triggers for libc-bin (2.35-0ubuntu3.6) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. N: Download is performed unsandboxed as root as file '/home/vagrant/wazuh-indexer_4.8.1-1_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) root@ubuntu22:/home/vagrant# ls -l /etc/wazuh-indexer/ total 56 -rw-rw---- 1 wazuh-indexer wazuh-indexer 2937 Jul 10 12:55 jvm.options drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jun 6 12:10 jvm.options.d -rw-r----- 1 wazuh-indexer wazuh-indexer 14808 Jun 6 12:10 log4j2.properties -rw-rw---- 1 wazuh-indexer wazuh-indexer 196 Jul 10 12:55 opensearch.keystore drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:56 opensearch-notifications drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:56 opensearch-notifications-core drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:56 opensearch-observability drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:56 opensearch-performance-analyzer drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:56 opensearch-reports-scheduler drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:56 opensearch-security -rw-rw---- 1 wazuh-indexer wazuh-indexer 2081 Jun 6 12:10 opensearch.yml ```
install 4.8.0 -> remove 4.8.0 (no purge)-> install 4.8.1 :green_circle: ``` root@ubuntu22:/home/vagrant# apt install ./wazuh-indexer_4.8.0-1_amd64.deb Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'wazuh-indexer' instead of './wazuh-indexer_4.8.0-1_amd64.deb' The following NEW packages will be installed: wazuh-indexer 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/759 MB of archives. After this operation, 1,050 MB of additional disk space will be used. Get:1 /home/vagrant/wazuh-indexer_4.8.0-1_amd64.deb wazuh-indexer amd64 4.8.0-1 [759 MB] Selecting previously unselected package wazuh-indexer. (Reading database ... 76324 files and directories currently installed.) Preparing to unpack .../wazuh-indexer_4.8.0-1_amd64.deb ... Creating wazuh-indexer group... OK Creating wazuh-indexer user... OK Unpacking wazuh-indexer (4.8.0-1) ... Setting up wazuh-indexer (4.8.0-1) ... Created opensearch keystore in /etc/wazuh-indexer/opensearch.keystore Processing triggers for libc-bin (2.35-0ubuntu3.6) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. N: Download is performed unsandboxed as root as file '/home/vagrant/wazuh-indexer_4.8.0-1_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) root@ubuntu22:/home/vagrant# apt remove wazuh-indexer -y Reading package lists... Done Building dependency tree... Done Reading state information... Done The following packages will be REMOVED: wazuh-indexer 0 upgraded, 0 newly installed, 1 to remove and 2 not upgraded. After this operation, 1,050 MB disk space will be freed. (Reading database ... 77497 files and directories currently installed.) Removing wazuh-indexer (4.8.0-1) ... Stopping wazuh-indexer service... OK root@ubuntu22:/home/vagrant# ls -l /etc/wazuh-indexer/ total 52 -rw-rw---- 1 wazuh-indexer wazuh-indexer 2937 Jul 10 12:58 jvm.options -rw-r----- 1 wazuh-indexer wazuh-indexer 14808 Jun 6 12:10 log4j2.properties -rw-rw---- 1 wazuh-indexer wazuh-indexer 196 Jul 10 12:58 opensearch.keystore drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:58 opensearch-notifications drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:58 opensearch-notifications-core drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:58 opensearch-observability drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:58 opensearch-performance-analyzer drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:58 opensearch-reports-scheduler drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:58 opensearch-security -rw-rw---- 1 wazuh-indexer wazuh-indexer 2081 Jun 6 12:10 opensearch.yml root@ubuntu22:/home/vagrant# apt install ./wazuh-indexer_4.8.1-1_amd64.deb Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'wazuh-indexer' instead of './wazuh-indexer_4.8.1-1_amd64.deb' The following NEW packages will be installed: wazuh-indexer 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/759 MB of archives. After this operation, 1,050 MB of additional disk space will be used. Get:1 /home/vagrant/wazuh-indexer_4.8.1-1_amd64.deb wazuh-indexer amd64 4.8.1-1 [759 MB] Selecting previously unselected package wazuh-indexer. (Reading database ... 76362 files and directories currently installed.) Preparing to unpack .../wazuh-indexer_4.8.1-1_amd64.deb ... Unpacking wazuh-indexer (4.8.1-1) ... Setting up wazuh-indexer (4.8.1-1) ... Processing triggers for libc-bin (2.35-0ubuntu3.6) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. N: Download is performed unsandboxed as root as file '/home/vagrant/wazuh-indexer_4.8.1-1_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) root@ubuntu22:/home/vagrant# ls -l /etc/wazuh-indexer/ total 56 -rw-rw---- 1 wazuh-indexer wazuh-indexer 2937 Jul 10 12:58 jvm.options drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 4 21:17 jvm.options.d -rw-r----- 1 wazuh-indexer wazuh-indexer 14808 Jun 6 12:10 log4j2.properties -rw-rw---- 1 wazuh-indexer wazuh-indexer 196 Jul 10 12:58 opensearch.keystore drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:59 opensearch-notifications drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:59 opensearch-notifications-core drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:59 opensearch-observability drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:59 opensearch-performance-analyzer drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:59 opensearch-reports-scheduler drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 12:59 opensearch-security -rw-rw---- 1 wazuh-indexer wazuh-indexer 2081 Jun 6 12:10 opensearch.yml ```
install 4.8.0 -> remove etc -> remove 4.8.0 (no purge)-> install 4.8.1 :red_circle: ``` root@ubuntu22:/home/vagrant# apt install ./wazuh-indexer_4.8.0-1_amd64.deb -y Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'wazuh-indexer' instead of './wazuh-indexer_4.8.0-1_amd64.deb' The following NEW packages will be installed: wazuh-indexer 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/759 MB of archives. After this operation, 1,050 MB of additional disk space will be used. Get:1 /home/vagrant/wazuh-indexer_4.8.0-1_amd64.deb wazuh-indexer amd64 4.8.0-1 [759 MB] Selecting previously unselected package wazuh-indexer. (Reading database ... 76324 files and directories currently installed.) Preparing to unpack .../wazuh-indexer_4.8.0-1_amd64.deb ... Creating wazuh-indexer group... OK Creating wazuh-indexer user... OK Unpacking wazuh-indexer (4.8.0-1) ... Setting up wazuh-indexer (4.8.0-1) ... Created opensearch keystore in /etc/wazuh-indexer/opensearch.keystore Processing triggers for libc-bin (2.35-0ubuntu3.6) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. N: Download is performed unsandboxed as root as file '/home/vagrant/wazuh-indexer_4.8.0-1_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) root@ubuntu22:/home/vagrant# ls -l /etc/wazuh-indexer/ total 56 -rw-rw---- 1 wazuh-indexer wazuh-indexer 2937 Jul 10 13:39 jvm.options drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jun 6 12:10 jvm.options.d -rw-r----- 1 wazuh-indexer wazuh-indexer 14808 Jun 6 12:10 log4j2.properties -rw-rw---- 1 wazuh-indexer wazuh-indexer 196 Jul 10 13:39 opensearch.keystore drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:39 opensearch-notifications drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:39 opensearch-notifications-core drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:39 opensearch-observability drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:39 opensearch-performance-analyzer drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:39 opensearch-reports-scheduler drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:39 opensearch-security -rw-rw---- 1 wazuh-indexer wazuh-indexer 2081 Jun 6 12:10 opensearch.yml root@ubuntu22:/home/vagrant# rm -rf /etc/wazuh-indexer root@ubuntu22:/home/vagrant# apt remove wazuh-indexer -y Reading package lists... Done Building dependency tree... Done Reading state information... Done The following packages will be REMOVED: wazuh-indexer 0 upgraded, 0 newly installed, 1 to remove and 2 not upgraded. After this operation, 1,050 MB disk space will be freed. (Reading database ... 77497 files and directories currently installed.) Removing wazuh-indexer (4.8.0-1) ... Stopping wazuh-indexer service... OK root@ubuntu22:/home/vagrant# dpkg -L wazuh-indexer /usr /usr/lib /usr/lib/systemd /usr/lib/systemd/system /var /var/lib /var/lib/wazuh-indexer /etc /etc/init.d /etc/init.d/wazuh-indexer /etc/wazuh-indexer/opensearch-observability/observability.yml /etc/wazuh-indexer/opensearch-security/roles_mapping.yml /etc/wazuh-indexer/opensearch-security/nodes_dn.yml /etc/wazuh-indexer/opensearch-security/internal_users.yml /etc/wazuh-indexer/opensearch-security/roles.yml /etc/wazuh-indexer/opensearch-security/allowlist.yml /etc/wazuh-indexer/opensearch-security/audit.yml /etc/wazuh-indexer/opensearch-security/whitelist.yml /etc/wazuh-indexer/opensearch-security/tenants.yml /etc/wazuh-indexer/opensearch-security/config.yml /etc/wazuh-indexer/opensearch-security/action_groups.yml /etc/wazuh-indexer/opensearch-security/opensearch.yml.example /etc/wazuh-indexer/log4j2.properties /etc/wazuh-indexer/opensearch-notifications-core/notifications-core.yml /etc/wazuh-indexer/opensearch-reports-scheduler/reports-scheduler.yml /etc/wazuh-indexer/opensearch-performance-analyzer/rca_idle_cluster_manager.conf /etc/wazuh-indexer/opensearch-performance-analyzer/supervisord.conf /etc/wazuh-indexer/opensearch-performance-analyzer/performance-analyzer.properties /etc/wazuh-indexer/opensearch-performance-analyzer/log4j2.xml /etc/wazuh-indexer/opensearch-performance-analyzer/plugin-stats-metadata /etc/wazuh-indexer/opensearch-performance-analyzer/opensearch_security.policy /etc/wazuh-indexer/opensearch-performance-analyzer/rca.conf /etc/wazuh-indexer/opensearch-performance-analyzer/rca_cluster_manager.conf /etc/wazuh-indexer/opensearch-performance-analyzer/agent-stats-metadata /etc/wazuh-indexer/jvm.options /etc/wazuh-indexer/opensearch-notifications/notifications.yml /etc/wazuh-indexer/opensearch.yml /etc/default /etc/default/wazuh-indexer root@ubuntu22:/home/vagrant# apt install ./wazuh-indexer_4.8.1-1_amd64.deb Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'wazuh-indexer' instead of './wazuh-indexer_4.8.1-1_amd64.deb' The following NEW packages will be installed: wazuh-indexer 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/759 MB of archives. After this operation, 1,050 MB of additional disk space will be used. Get:1 /home/vagrant/wazuh-indexer_4.8.1-1_amd64.deb wazuh-indexer amd64 4.8.1-1 [759 MB] Selecting previously unselected package wazuh-indexer. (Reading database ... 76355 files and directories currently installed.) Preparing to unpack .../wazuh-indexer_4.8.1-1_amd64.deb ... Unpacking wazuh-indexer (4.8.1-1) ... Setting up wazuh-indexer (4.8.1-1) ... Created opensearch keystore in /etc/wazuh-indexer/opensearch.keystore Processing triggers for libc-bin (2.35-0ubuntu3.6) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. N: Download is performed unsandboxed as root as file '/home/vagrant/wazuh-indexer_4.8.1-1_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) root@ubuntu22:/home/vagrant# ls -l /etc/wazuh-indexer/ total 32 drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 4 21:17 jvm.options.d -rw-rw---- 1 wazuh-indexer wazuh-indexer 196 Jul 10 13:40 opensearch.keystore drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:40 opensearch-notifications drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:40 opensearch-notifications-core drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:40 opensearch-observability drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:40 opensearch-performance-analyzer drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:40 opensearch-reports-scheduler drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:40 opensearch-security ```
install 4.8.0 -> remove etc -> remove 4.8.0 (purge)-> install 4.8.1 :green_circle: ``` root@ubuntu22:/home/vagrant# apt install ./wazuh-indexer_4.8.0-1_amd64.deb -y Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'wazuh-indexer' instead of './wazuh-indexer_4.8.0-1_amd64.deb' The following NEW packages will be installed: wazuh-indexer 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/759 MB of archives. After this operation, 1,050 MB of additional disk space will be used. Get:1 /home/vagrant/wazuh-indexer_4.8.0-1_amd64.deb wazuh-indexer amd64 4.8.0-1 [759 MB] Selecting previously unselected package wazuh-indexer. (Reading database ... 76324 files and directories currently installed.) Preparing to unpack .../wazuh-indexer_4.8.0-1_amd64.deb ... Creating wazuh-indexer group... OK Creating wazuh-indexer user... OK Unpacking wazuh-indexer (4.8.0-1) ... Setting up wazuh-indexer (4.8.0-1) ... Created opensearch keystore in /etc/wazuh-indexer/opensearch.keystore Processing triggers for libc-bin (2.35-0ubuntu3.6) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. N: Download is performed unsandboxed as root as file '/home/vagrant/wazuh-indexer_4.8.0-1_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) root@ubuntu22:/home/vagrant# rm -rf /etc/wazuh-indexer root@ubuntu22:/home/vagrant# apt remove --purge wazuh-indexer -y Reading package lists... Done Building dependency tree... Done Reading state information... Done The following packages will be REMOVED: wazuh-indexer* 0 upgraded, 0 newly installed, 1 to remove and 2 not upgraded. After this operation, 1,050 MB disk space will be freed. (Reading database ... 77497 files and directories currently installed.) Removing wazuh-indexer (4.8.0-1) ... Stopping wazuh-indexer service... OK (Reading database ... 76355 files and directories currently installed.) Purging configuration files for wazuh-indexer (4.8.0-1) ... Deleting configuration directory... OK dpkg: warning: while removing wazuh-indexer, directory '/var/lib/wazuh-indexer' not empty so not removed dpkg: warning: while removing wazuh-indexer, directory '/usr/lib/systemd/system' not empty so not removed root@ubuntu22:/home/vagrant# dpkg -L wazuh-indexer dpkg-query: package 'wazuh-indexer' is not installed Use dpkg --contents (= dpkg-deb --contents) to list archive files contents. root@ubuntu22:/home/vagrant# apt install ./wazuh-indexer_4.8.1-1_amd64.deb -y Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'wazuh-indexer' instead of './wazuh-indexer_4.8.1-1_amd64.deb' The following NEW packages will be installed: wazuh-indexer 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/759 MB of archives. After this operation, 1,050 MB of additional disk space will be used. Get:1 /home/vagrant/wazuh-indexer_4.8.1-1_amd64.deb wazuh-indexer amd64 4.8.1-1 [759 MB] Selecting previously unselected package wazuh-indexer. (Reading database ... 76324 files and directories currently installed.) Preparing to unpack .../wazuh-indexer_4.8.1-1_amd64.deb ... Creating wazuh-indexer group... OK Creating wazuh-indexer user... OK Unpacking wazuh-indexer (4.8.1-1) ... Setting up wazuh-indexer (4.8.1-1) ... Created opensearch keystore in /etc/wazuh-indexer/opensearch.keystore Processing triggers for libc-bin (2.35-0ubuntu3.6) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. N: Download is performed unsandboxed as root as file '/home/vagrant/wazuh-indexer_4.8.1-1_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) root@ubuntu22:/home/vagrant# ls -l /etc/wazuh-indexer/ total 56 -rw-rw---- 1 wazuh-indexer wazuh-indexer 2937 Jul 10 13:43 jvm.options drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 4 21:17 jvm.options.d -rw-r----- 1 wazuh-indexer wazuh-indexer 14808 Jul 4 21:17 log4j2.properties -rw-rw---- 1 wazuh-indexer wazuh-indexer 196 Jul 10 13:43 opensearch.keystore drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:43 opensearch-notifications drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:43 opensearch-notifications-core drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:43 opensearch-observability drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:43 opensearch-performance-analyzer drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:43 opensearch-reports-scheduler drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 13:43 opensearch-security -rw-rw---- 1 wazuh-indexer wazuh-indexer 2081 Jul 4 21:17 opensearch.yml ```

Note that the only case where the installation failed is when the previous package was removed from the system without a purge

Removing installed files from a package broke the package and the system DDBB, as it can be seen, after removal, the system still has those /etc/wazuh-indexer files indexed in the DDBB

root@ubuntu22:/home/vagrant# dpkg -L wazuh-indexer
/usr
/usr/lib
/usr/lib/systemd
/usr/lib/systemd/system
/var
/var/lib
/var/lib/wazuh-indexer
/etc
/etc/init.d
/etc/init.d/wazuh-indexer
/etc/wazuh-indexer/opensearch-observability/observability.yml
/etc/wazuh-indexer/opensearch-security/roles_mapping.yml
/etc/wazuh-indexer/opensearch-security/nodes_dn.yml
/etc/wazuh-indexer/opensearch-security/internal_users.yml
/etc/wazuh-indexer/opensearch-security/roles.yml
/etc/wazuh-indexer/opensearch-security/allowlist.yml
/etc/wazuh-indexer/opensearch-security/audit.yml
/etc/wazuh-indexer/opensearch-security/whitelist.yml
/etc/wazuh-indexer/opensearch-security/tenants.yml
/etc/wazuh-indexer/opensearch-security/config.yml
/etc/wazuh-indexer/opensearch-security/action_groups.yml
/etc/wazuh-indexer/opensearch-security/opensearch.yml.example
/etc/wazuh-indexer/log4j2.properties
/etc/wazuh-indexer/opensearch-notifications-core/notifications-core.yml
/etc/wazuh-indexer/opensearch-reports-scheduler/reports-scheduler.yml
/etc/wazuh-indexer/opensearch-performance-analyzer/rca_idle_cluster_manager.conf
/etc/wazuh-indexer/opensearch-performance-analyzer/supervisord.conf
/etc/wazuh-indexer/opensearch-performance-analyzer/performance-analyzer.properties
/etc/wazuh-indexer/opensearch-performance-analyzer/log4j2.xml
/etc/wazuh-indexer/opensearch-performance-analyzer/plugin-stats-metadata
/etc/wazuh-indexer/opensearch-performance-analyzer/opensearch_security.policy
/etc/wazuh-indexer/opensearch-performance-analyzer/rca.conf
/etc/wazuh-indexer/opensearch-performance-analyzer/rca_cluster_manager.conf
/etc/wazuh-indexer/opensearch-performance-analyzer/agent-stats-metadata
/etc/wazuh-indexer/jvm.options
/etc/wazuh-indexer/opensearch-notifications/notifications.yml
/etc/wazuh-indexer/opensearch.yml
/etc/default
/etc/default/wazuh-indexer

The apt remove command without the purge option does not clean the DDBB, and the system assumes that those files are still present in the system (they are not as they have been removed manually)

Note that this does not only happen with the opensearch.yml file but also with other files like log4j2.properties and jvm.options, the only existing file is opensearch.keystore because we create it if not exist due to the username and password https://github.com/wazuh/wazuh-packages/blob/ddaaabf81b1c0509656ffa1df29517184ea8bcb2/stack/indexer/deb/debian/postinst#L119-L125

In the 4.7.5 -> 4.8.0 upgrade, the error is the same, the only difference is that the package manager detects that some files are different (checksum check), and as the default value is N, the Jenkins build reported in the referenced issue keeps the current system file (the files do not exist as it was removed manually but the DDBB still thinks that it exists), then, the package is upgraded but the file is maintained, and as the file does not exist, is missing after the upgrade

Configuration file '/etc/wazuh-indexer/opensearch-security/roles_mapping.yml'
 ==> Deleted (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** roles_mapping.yml (Y/I/N/O/D/Z) [default=N] ?
install 4.7.5 -> remove etc -> remove 4.7.5 (no purge)-> install 4.8.0 :red_circle: ``` root@ubuntu22:/home/vagrant# apt install ./wazuh-indexer_4.7.5-1_amd64.deb -y Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'wazuh-indexer' instead of './wazuh-indexer_4.7.5-1_amd64.deb' The following NEW packages will be installed: wazuh-indexer 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/686 MB of archives. After this operation, 969 MB of additional disk space will be used. Get:1 /home/vagrant/wazuh-indexer_4.7.5-1_amd64.deb wazuh-indexer amd64 4.7.5-1 [686 MB] Selecting previously unselected package wazuh-indexer. (Reading database ... 76324 files and directories currently installed.) Preparing to unpack .../wazuh-indexer_4.7.5-1_amd64.deb ... Creating wazuh-indexer group... OK Creating wazuh-indexer user... OK Unpacking wazuh-indexer (4.7.5-1) ... Setting up wazuh-indexer (4.7.5-1) ... Created opensearch keystore in /etc/wazuh-indexer/opensearch.keystore Processing triggers for libc-bin (2.35-0ubuntu3.6) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. N: Download is performed unsandboxed as root as file '/home/vagrant/wazuh-indexer_4.7.5-1_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) root@ubuntu22:/home/vagrant# ls -l /etc/wazuh-indexer/ total 56 -rw-rw---- 1 wazuh-indexer wazuh-indexer 2937 Jul 10 14:17 jvm.options drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 May 29 16:43 jvm.options.d -rw-r----- 1 wazuh-indexer wazuh-indexer 14808 May 29 16:43 log4j2.properties -rw-rw---- 1 wazuh-indexer wazuh-indexer 196 Jul 10 14:17 opensearch.keystore drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 14:17 opensearch-notifications drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 14:17 opensearch-notifications-core drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 14:17 opensearch-observability drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 14:17 opensearch-performance-analyzer drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 14:17 opensearch-reports-scheduler drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 14:17 opensearch-security -rw-rw---- 1 wazuh-indexer wazuh-indexer 2081 May 29 16:43 opensearch.yml root@ubuntu22:/home/vagrant# rm -rf /etc/wazuh-indexer/ root@ubuntu22:/home/vagrant# apt remove wazuh-indexer Reading package lists... Done Building dependency tree... Done Reading state information... Done The following packages will be REMOVED: wazuh-indexer 0 upgraded, 0 newly installed, 1 to remove and 2 not upgraded. After this operation, 969 MB disk space will be freed. Do you want to continue? [Y/n] y (Reading database ... 77451 files and directories currently installed.) Removing wazuh-indexer (4.7.5-1) ... Stopping wazuh-indexer service... OK root@ubuntu22:/home/vagrant# dpkg -L wazuh-indexer /usr /usr/lib /usr/lib/systemd /usr/lib/systemd/system /var /var/lib /var/lib/wazuh-indexer /etc /etc/init.d /etc/init.d/wazuh-indexer /etc/wazuh-indexer/opensearch-observability/observability.yml /etc/wazuh-indexer/opensearch-security/roles_mapping.yml /etc/wazuh-indexer/opensearch-security/nodes_dn.yml /etc/wazuh-indexer/opensearch-security/internal_users.yml /etc/wazuh-indexer/opensearch-security/roles.yml /etc/wazuh-indexer/opensearch-security/allowlist.yml /etc/wazuh-indexer/opensearch-security/audit.yml /etc/wazuh-indexer/opensearch-security/whitelist.yml /etc/wazuh-indexer/opensearch-security/tenants.yml /etc/wazuh-indexer/opensearch-security/config.yml /etc/wazuh-indexer/opensearch-security/action_groups.yml /etc/wazuh-indexer/opensearch-security/opensearch.yml.example /etc/wazuh-indexer/log4j2.properties /etc/wazuh-indexer/opensearch-notifications-core/notifications-core.yml /etc/wazuh-indexer/opensearch-reports-scheduler/reports-scheduler.yml /etc/wazuh-indexer/opensearch-performance-analyzer/rca_idle_cluster_manager.conf /etc/wazuh-indexer/opensearch-performance-analyzer/supervisord.conf /etc/wazuh-indexer/opensearch-performance-analyzer/performance-analyzer.properties /etc/wazuh-indexer/opensearch-performance-analyzer/log4j2.xml /etc/wazuh-indexer/opensearch-performance-analyzer/plugin-stats-metadata /etc/wazuh-indexer/opensearch-performance-analyzer/opensearch_security.policy /etc/wazuh-indexer/opensearch-performance-analyzer/rca.conf /etc/wazuh-indexer/opensearch-performance-analyzer/rca_cluster_manager.conf /etc/wazuh-indexer/opensearch-performance-analyzer/agent-stats-metadata /etc/wazuh-indexer/jvm.options /etc/wazuh-indexer/opensearch-notifications/notifications.yml /etc/wazuh-indexer/opensearch.yml /etc/default /etc/default/wazuh-indexer root@ubuntu22:/home/vagrant# apt install ./wazuh-indexer_4.8.0-1_amd64.deb -y Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'wazuh-indexer' instead of './wazuh-indexer_4.8.0-1_amd64.deb' The following NEW packages will be installed: wazuh-indexer 0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded. Need to get 0 B/759 MB of archives. After this operation, 1,050 MB of additional disk space will be used. Get:1 /home/vagrant/wazuh-indexer_4.8.0-1_amd64.deb wazuh-indexer amd64 4.8.0-1 [759 MB] Selecting previously unselected package wazuh-indexer. (Reading database ... 76355 files and directories currently installed.) Preparing to unpack .../wazuh-indexer_4.8.0-1_amd64.deb ... Unpacking wazuh-indexer (4.8.0-1) ... Setting up wazuh-indexer (4.8.0-1) ... Configuration file '/etc/wazuh-indexer/opensearch-notifications-core/notifications-core.yml' ==> Deleted (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it ? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. *** notifications-core.yml (Y/I/N/O/D/Z) [default=N] ? Configuration file '/etc/wazuh-indexer/opensearch-performance-analyzer/rca.conf' ==> Deleted (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it ? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. *** rca.conf (Y/I/N/O/D/Z) [default=N] ? Configuration file '/etc/wazuh-indexer/opensearch-performance-analyzer/rca_cluster_manager.conf' ==> Deleted (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it ? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. *** rca_cluster_manager.conf (Y/I/N/O/D/Z) [default=N] ? Configuration file '/etc/wazuh-indexer/opensearch-performance-analyzer/rca_idle_cluster_manager.conf' ==> Deleted (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it ? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. *** rca_idle_cluster_manager.conf (Y/I/N/O/D/Z) [default=N] ? Configuration file '/etc/wazuh-indexer/opensearch-performance-analyzer/supervisord.conf' ==> Deleted (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it ? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. *** supervisord.conf (Y/I/N/O/D/Z) [default=N] ? Configuration file '/etc/wazuh-indexer/opensearch-security/roles.yml' ==> Deleted (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it ? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. *** roles.yml (Y/I/N/O/D/Z) [default=N] ? Configuration file '/etc/wazuh-indexer/opensearch-security/roles_mapping.yml' ==> Deleted (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it ? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. *** roles_mapping.yml (Y/I/N/O/D/Z) [default=N] ? Created opensearch keystore in /etc/wazuh-indexer/opensearch.keystore Processing triggers for libc-bin (2.35-0ubuntu3.6) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. N: Download is performed unsandboxed as root as file '/home/vagrant/wazuh-indexer_4.8.0-1_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) root@ubuntu22:/home/vagrant# ls -l /etc/wazuh-indexer/ total 32 drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jun 6 12:10 jvm.options.d -rw-rw---- 1 wazuh-indexer wazuh-indexer 196 Jul 10 14:20 opensearch.keystore drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 14:20 opensearch-notifications drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 14:20 opensearch-notifications-core drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 14:20 opensearch-observability drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 14:20 opensearch-performance-analyzer drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 14:20 opensearch-reports-scheduler drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Jul 10 14:20 opensearch-security ```

Conclusion

This is an expected error related to the system package management, we can't control it from the package itself (we can't do anything if someone removes files deliberately from the system), and this is the purpose of the purge option

juliamagan commented 4 months ago

LGTM