search

wazuh / wazuh-packages

Wazuh - Tools for packages creation
https://wazuh.com
GNU General Public License v2.0
104 stars 94 forks source link

Indexer template creation failed #2770

Closed juliamagan closed 8 months ago

juliamagan commented 8 months ago
Wazuh version Install type Action performed Platform
v4.8.0-alpha2 Indexer Install Ubuntu 20.04

Description

During the tests performed in https://github.com/wazuh/wazuh/issues/21365, it was found that when using the offline installation of Wazuh and initializing the indexer, the templates could not be created:

  Executing Wazuh indexer ISM init script...
  Will create 'wazuh' index template
    ERROR: 'wazuh' template creation failed
  Will create 'ism_history_indices' index template
    ERROR: 'ism_history_indices' template creation failed
  Will disable replicas for 'plugins.index_state_management.history' indices
    ERROR: cluster's settings update failed
  Will create index templates to configure the alias
    ERROR: 'wazuh-alerts' template creation failed
    ERROR: 'wazuh-archives' template creation failed
  ERROR: Indexer ISM initialization failed. Check /tmp/wazuh-indexer/ism-init.log for more information.

After a brief investigation, it was found that this is because the indexer-ism-init.sh script has the hostname hardcoded to localhost:

https://github.com/wazuh/wazuh-packages/blob/e5be5560407579f374fb41400127e7d187ca1588/stack/indexer/indexer-ism-init.sh#L12

This means that if in config.yml you set an IP or hostname other than localhost, it cannot initialize.

If we change the hostname to the IP used in the configuration, it seems to work correctly:

Executing Wazuh indexer ISM init script...
Will create 'wazuh' index template
 SUCC: 'wazuh' template created or updated
Will create 'ism_history_indices' index template
 SUCC: 'ism_history_indices' template created or updated
Will disable replicas for 'plugins.index_state_management.history' indices
 SUCC: cluster's settings saved
Will create index templates to configure the alias
 SUCC: 'wazuh-alerts' template created or updated
 SUCC: 'wazuh-archives' template created or updated
Will create the 'rollover_policy' policy
  SUCC: 'rollover_policy' policy created
Will create initial indices for the aliases
  SUCC: 'wazuh-alerts' write index created
  SUCC: 'wazuh-archives' write index created
SUCC: Indexer ISM initialization finished successfully.

Also, it says that you can see more information in the log, but the file does not exist:

  root@ubuntu-indexer:/home/vagrant# cat /tmp/wazuh-indexer/ism-init.log
  cat: /tmp/wazuh-indexer/ism-init.log: No such file or directory
AlexRuiz7 commented 8 months ago

The hostname is not hard-coded to localhost, that's the default value in case no value was provided.

To specify the hostname, use the -i or --indexer-hostname option, as stated in the documentation.

bash /usr/share/wazuh-indexer/bin/indexer-init.sh -i <WAZUH_INDEXER_IP_ADDRESS>

Looks like that is missing in the offline installation documentation:

/usr/share/wazuh-indexer/bin/indexer-init.sh

We'll fix that.

@juliamagan can you please confirm that using the -i option successfully initializes the cluster?

juliamagan commented 8 months ago

It works as expected with the command that @AlexRuiz7 provided:

root@ubuntu-indexer:/home/vagrant# /usr/share/wazuh-indexer/bin/indexer-init.sh -i 172.17.1.21
Executing Wazuh indexer security init script...
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
Security Admin v7
Will connect to 172.17.1.21:9200 ... done
Connected as "CN=admin,OU=Wazuh,O=Wazuh,L=California,C=US"
OpenSearch Version: 2.10.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: wazuh-cluster
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index already exists, so we do not need to create one.
Populate config from /etc/wazuh-indexer/opensearch-security/
Will update '/config' with /etc/wazuh-indexer/opensearch-security/config.yml 
   SUCC: Configuration for 'config' created or updated
Will update '/roles' with /etc/wazuh-indexer/opensearch-security/roles.yml 
   SUCC: Configuration for 'roles' created or updated
Will update '/rolesmapping' with /etc/wazuh-indexer/opensearch-security/roles_mapping.yml 
   SUCC: Configuration for 'rolesmapping' created or updated
Will update '/internalusers' with /etc/wazuh-indexer/opensearch-security/internal_users.yml 
   SUCC: Configuration for 'internalusers' created or updated
Will update '/actiongroups' with /etc/wazuh-indexer/opensearch-security/action_groups.yml 
   SUCC: Configuration for 'actiongroups' created or updated
Will update '/tenants' with /etc/wazuh-indexer/opensearch-security/tenants.yml 
   SUCC: Configuration for 'tenants' created or updated
Will update '/nodesdn' with /etc/wazuh-indexer/opensearch-security/nodes_dn.yml 
   SUCC: Configuration for 'nodesdn' created or updated
Will update '/whitelist' with /etc/wazuh-indexer/opensearch-security/whitelist.yml 
   SUCC: Configuration for 'whitelist' created or updated
Will update '/audit' with /etc/wazuh-indexer/opensearch-security/audit.yml 
   SUCC: Configuration for 'audit' created or updated
Will update '/allowlist' with /etc/wazuh-indexer/opensearch-security/allowlist.yml 
   SUCC: Configuration for 'allowlist' created or updated
SUCC: Expected 10 config types for node {"updated_config_types":["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"],"updated_config_size":10,"message":null} is 10 (["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"]) due to: null
Done with success
Executing Wazuh indexer ISM init script...
Will create 'wazuh' index template
 SUCC: 'wazuh' template created or updated
Will create 'ism_history_indices' index template
 SUCC: 'ism_history_indices' template created or updated
Will disable replicas for 'plugins.index_state_management.history' indices
 SUCC: cluster's settings saved
Will create index templates to configure the alias
 SUCC: 'wazuh-alerts' template created or updated
 SUCC: 'wazuh-archives' template created or updated
Will create the 'rollover_policy' policy
  INFO: policy 'rollover_policy' already exists. Skipping policy creation
Will create initial indices for the aliases
  INFO: 'wazuh-alerts' write index already exists. Skipping write index creation
  INFO: 'wazuh-archives' write index already exists. Skipping write index creation
SUCC: Indexer ISM initialization finished successfully.
AlexRuiz7 commented 8 months ago

Currently investigating the problem with the log file

AlexRuiz7 commented 8 months ago

Unable to reproduce

mhamra commented 8 months ago

UPDATE

I found the same problem doing the E2E test https://github.com/wazuh/wazuh/issues/21767 for v4.8.0-beta-1.

I've used the -i parameter with the value node-1 (the indexer's name in config.yml). The name was resolved OK to the indexer's IP address, but the command did not succeed.

root@indexer-1:/home/vagrant# bash /usr/share/wazuh-indexer/bin/indexer-init.sh -i node-1
Executing Wazuh indexer security init script...
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
Security Admin v7
Will connect to 192.168.56.8:9200 ... done
Connected as "CN=admin,OU=Wazuh,O=Wazuh,L=California,C=US"
OpenSearch Version: 2.10.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: wazuh-cluster
Clusterstate: GREEN
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index does not exists, attempt to create it ... done (0-all replicas)
Populate config from /etc/wazuh-indexer/opensearch-security/
Will update '/config' with /etc/wazuh-indexer/opensearch-security/config.yml
   SUCC: Configuration for 'config' created or updated
Will update '/roles' with /etc/wazuh-indexer/opensearch-security/roles.yml
   SUCC: Configuration for 'roles' created or updated
Will update '/rolesmapping' with /etc/wazuh-indexer/opensearch-security/roles_mapping.yml
   SUCC: Configuration for 'rolesmapping' created or updated
Will update '/internalusers' with /etc/wazuh-indexer/opensearch-security/internal_users.yml
   SUCC: Configuration for 'internalusers' created or updated
Will update '/actiongroups' with /etc/wazuh-indexer/opensearch-security/action_groups.yml
   SUCC: Configuration for 'actiongroups' created or updated
Will update '/tenants' with /etc/wazuh-indexer/opensearch-security/tenants.yml
   SUCC: Configuration for 'tenants' created or updated
Will update '/nodesdn' with /etc/wazuh-indexer/opensearch-security/nodes_dn.yml
   SUCC: Configuration for 'nodesdn' created or updated
Will update '/whitelist' with /etc/wazuh-indexer/opensearch-security/whitelist.yml
   SUCC: Configuration for 'whitelist' created or updated
Will update '/audit' with /etc/wazuh-indexer/opensearch-security/audit.yml
   SUCC: Configuration for 'audit' created or updated
Will update '/allowlist' with /etc/wazuh-indexer/opensearch-security/allowlist.yml
   SUCC: Configuration for 'allowlist' created or updated
SUCC: Expected 10 config types for node {"updated_config_types":["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"],"updated_config_size":10,"message":null} is 10 (["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"]) due to: null
Done with success
Executing Wazuh indexer ISM init script...
Will create 'wazuh' index template
  ERROR: 'wazuh' template creation failed
Will create 'ism_history_indices' index template
  ERROR: 'ism_history_indices' template creation failed
Will disable replicas for 'plugins.index_state_management.history' indices
  ERROR: cluster's settings update failed
Will create index templates to configure the alias
  ERROR: 'wazuh-alerts' template creation failed
  ERROR: 'wazuh-archives' template creation failed
ERROR: Indexer ISM initialization failed. Check /tmp/wazuh-indexer/ism-init.log for more information.

I repeated this command after doing all the steps of the E2E, and it finished successfully.

root@indexer-1:/home/vagrant# bash /usr/share/wazuh-indexer/bin/indexer-init.sh -i 192.168.56.8
Executing Wazuh indexer security init script...
**************************************************************************
** This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
Security Admin v7
Will connect to 192.168.56.8:9200 ... done
Connected as "CN=admin,OU=Wazuh,O=Wazuh,L=California,C=US"
OpenSearch Version: 2.10.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: wazuh-cluster
Clusterstate: YELLOW
Number of nodes: 1
Number of data nodes: 1
.opendistro_security index already exists, so we do not need to create one.
Populate config from /etc/wazuh-indexer/opensearch-security/
Will update '/config' with /etc/wazuh-indexer/opensearch-security/config.yml
   SUCC: Configuration for 'config' created or updated
Will update '/roles' with /etc/wazuh-indexer/opensearch-security/roles.yml
   SUCC: Configuration for 'roles' created or updated
Will update '/rolesmapping' with /etc/wazuh-indexer/opensearch-security/roles_mapping.yml
   SUCC: Configuration for 'rolesmapping' created or updated
Will update '/internalusers' with /etc/wazuh-indexer/opensearch-security/internal_users.yml
   SUCC: Configuration for 'internalusers' created or updated
Will update '/actiongroups' with /etc/wazuh-indexer/opensearch-security/action_groups.yml
   SUCC: Configuration for 'actiongroups' created or updated
Will update '/tenants' with /etc/wazuh-indexer/opensearch-security/tenants.yml
   SUCC: Configuration for 'tenants' created or updated
Will update '/nodesdn' with /etc/wazuh-indexer/opensearch-security/nodes_dn.yml
   SUCC: Configuration for 'nodesdn' created or updated
Will update '/whitelist' with /etc/wazuh-indexer/opensearch-security/whitelist.yml
   SUCC: Configuration for 'whitelist' created or updated
Will update '/audit' with /etc/wazuh-indexer/opensearch-security/audit.yml
   SUCC: Configuration for 'audit' created or updated
Will update '/allowlist' with /etc/wazuh-indexer/opensearch-security/allowlist.yml
   SUCC: Configuration for 'allowlist' created or updated
SUCC: Expected 10 config types for node {"updated_config_types":["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"],"updated_config_size":10,"message":null} is 10 (["allowlist","tenants","rolesmapping","nodesdn","audit","roles","whitelist","internalusers","actiongroups","config"]) due to: null
Done with success
Executing Wazuh indexer ISM init script...
Will create 'wazuh' index template
 SUCC: 'wazuh' template created or updated
Will create 'ism_history_indices' index template
 SUCC: 'ism_history_indices' template created or updated
Will disable replicas for 'plugins.index_state_management.history' indices
 SUCC: cluster's settings saved
Will create index templates to configure the alias
 SUCC: 'wazuh-alerts' template created or updated
 SUCC: 'wazuh-archives' template created or updated
Will create the 'rollover_policy' policy
  SUCC: 'rollover_policy' policy created
Will create initial indices for the aliases
  SUCC: 'wazuh-alerts' write index created
  SUCC: 'wazuh-archives' write index created
SUCC: Indexer ISM initialization finished successfully.
AlexRuiz7 commented 8 months ago

@mhamra have you checked the logs at /tmp/wazuh-indexer/ism-init.log ??

mhamra commented 8 months ago

@AlexRuiz7 I couldn't find the log file the indexer-init.sh script mentioned. It's the same behavior reported by @juliamagan.

AlexRuiz7 commented 8 months ago

We could not reproduce the problem.

See evidences in https://github.com/wazuh/wazuh-indexer/issues/149