Closed vcerenu closed 2 years ago
vcerenu
commented
2 years ago Created the manifests for the installation of Wazuh Indexer within the branch https://github.com/wazuh/wazuh-puppet/tree/new-packages-release
manifests/wazuh_indexer.pp
manifests/params_wazuh_indexer.pp
templates/opensearch_yml.erbIt is necessary to carry out a deployment test with Wazuh Indexer packages
vcerenu
commented
2 years ago Added manifest for Wazuh Dashboard. The manifests were copied to a stable version (v4.2.5) the module was created and the installation tests of opendistro and kibana od were carried out, which I am taking as models for the creation of manifests.
During the installation I had the following errors
Notice: /Stage[main]/Wazuh::Kibana_od/Package[Installing OD Kibana...]/ensure: created
Notice: /Stage[main]/Wazuh::Kibana_od/File[Configure kibana.yml]/content:
--- /etc/kibana/kibana.yml 2021-04-05 20:51:54.000000000 +0000
+++ /tmp/puppet-file20220112-3992-dsrllc 2022-01-12 16:11:33.194395572 +0000
@@ -13,24 +13,16 @@
# Description:
# Default Kibana configuration for Open Distro.
+server.port: 5601
+server.host: 0.0.0.0
-elasticsearch.hosts: https://localhost:9200
+elasticsearch.hosts: ["https://localhost:9200"]
elasticsearch.ssl.verificationMode: none
-elasticsearch.username: kibanaserver
-elasticsearch.password: kibanaserver
+elasticsearch.username: admin
+elasticsearch.password: admin
elasticsearch.requestHeadersWhitelist: ["securitytenant","Authorization"]
-opendistro_security.multitenancy.enabled: true
-opendistro_security.multitenancy.tenants.preferred: ["Private", "Global"]
-opendistro_security.readonly_mode.roles: ["kibana_read_only"]
-
-# Use this setting if you are running kibana without https
-opendistro_security.cookie.secure: false
-newsfeed.enabled: false
-telemetry.optIn: false
-telemetry.enabled: false
-security.showInsecureClusterWarning: false
-# To configure a WMS map server for use with Kibana, see:
-# https://opendistro.github.io/for-elasticsearch-docs/docs/kibana/maptiles/
-map.includeElasticMapsService: false
+opendistro_security.multitenancy.enabled: false
+opendistro_security.multitenancy.tenants.preferred: ["Private", "Global"]
+opendistro_security.readonly_mode.roles: ["kibana_read_only"]
\ No newline at end of file
Info: Computing checksum on file /etc/kibana/kibana.yml
Info: /Stage[main]/Wazuh::Kibana_od/File[Configure kibana.yml]: Filebucketed /etc/kibana/kibana.yml to puppet with sum 0b69788e10c10be39589e61edcd74f77
Notice: /Stage[main]/Wazuh::Kibana_od/File[Configure kibana.yml]/content: content changed '{md5}0b69788e10c10be39589e61edcd74f77' to '{md5}ce82cce30102de427a2b7625a09918db'
Notice: /Stage[main]/Wazuh::Kibana_od/File[Configure kibana.yml]/owner: owner changed 'root' to 'kibana'
Notice: /Stage[main]/Wazuh::Kibana_od/File[Configure kibana.yml]/group: group changed 'root' to 'kibana'
Info: /Stage[main]/Wazuh::Kibana_od/File[Configure kibana.yml]: Scheduling refresh of Service[kibana]
Info: /Stage[main]/Wazuh::Kibana_od/File[Configure kibana.yml]: Scheduling refresh of Service[kibana]
Info: /Stage[main]/Wazuh::Kibana_od/File[Configure kibana.yml]: Scheduling refresh of Service[kibana]
Error: 'curl -u admin:admin -k -s -XGET https://localhost:9200' returned 7 instead of one of [0]
Error: /Stage[main]/Wazuh::Kibana_od/Exec[Waiting for opendistro elasticsearch...]/returns: change from 'notrun' to ['0'] failed: 'curl -u admin:admin -k -s -XGET https://localhost:9200' returned 7 instead of one of [0]
Notice: /Stage[main]/Wazuh::Kibana_od/Exec[Installing Wazuh App...]/returns: Attempting to transfer from https://packages.wazuh.com/wazuhapp/wazuhapp-4.2.5_7.10.2.zip
Notice: /Stage[main]/Wazuh::Kibana_od/Exec[Installing Wazuh App...]/returns: Attempting to transfer from https://artifacts.elastic.co/downloads/kibana-plugins/https://packages.wazuh.com/wazuhapp/wazuhapp-4.2.5_7.10.2.zip/https://packages.wazuh.com/wazuhapp/wazuhapp-4.2.5_7.10.2.zip-7.10.2.zip
Notice: /Stage[main]/Wazuh::Kibana_od/Exec[Installing Wazuh App...]/returns: Plugin installation was unsuccessful due to error "No valid url specified."
Error: 'sudo -u admin:admin -u kibana /usr/share/kibana/bin/kibana-plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-4.2.5_7.10.2.zip' returned 70 instead of one of [0]
Error: /Stage[main]/Wazuh::Kibana_od/Exec[Installing Wazuh App...]/returns: change from 'notrun' to ['0'] failed: 'sudo -u admin:admin -u kibana /usr/share/kibana/bin/kibana-plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-4.2.5_7.10.2.zip' returned 70 instead of one of [0]
Error: 'curl -u admin:admin -k -s -XDELETE -sL -I 'https://localhost:9200/.wazuh' -o /dev/null' returned 7 instead of one of [0]
Error: /Stage[main]/Wazuh::Kibana_od/Exec[Removing .wazuh index...]/returns: change from 'notrun' to ['0'] failed: 'curl -u admin:admin -k -s -XDELETE -sL -I 'https://localhost:9200/.wazuh' -o /dev/null' returned 7 instead of one of [0]
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /usr/share/kibana/plugins/wazuh/wazuh.yml20220112-3992-zupzc5.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/production/modules/wazuh/manifests/kibana_od.pp, line: 77)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /usr/share/kibana/plugins/wazuh/wazuh.yml20220112-3992-zupzc5.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/production/modules/wazuh/manifests/kibana_od.pp, line: 77)
Wrapped exception:
No such file or directory - A directory component in /usr/share/kibana/plugins/wazuh/wazuh.yml20220112-3992-zupzc5.lock does not exist or is a dangling symbolic link
Error: /Stage[main]/Wazuh::Kibana_od/File[/usr/share/kibana/plugins/wazuh/wazuh.yml]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: No such file or directory - A directory component in /usr/share/kibana/plugins/wazuh/wazuh.yml20220112-3992-zupzc5.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/production/modules/wazuh/manifests/kibana_od.pp, line: 77)
Notice: /Stage[main]/Wazuh::Kibana_od/Service[kibana]: Dependency Exec[Installing Wazuh App...] has failures: true
Notice: /Stage[main]/Wazuh::Kibana_od/Service[kibana]: Dependency Exec[Removing .wazuh index...] has failures: true
Notice: /Stage[main]/Wazuh::Kibana_od/Service[kibana]: Dependency File[/usr/share/kibana/plugins/wazuh/wazuh.yml] has failures: true
Warning: /Stage[main]/Wazuh::Kibana_od/Service[kibana]: Skipping because of failed dependencies
Info: /Stage[main]/Wazuh::Kibana_od/Service[kibana]: Unscheduling all events on Service[kibana]
Notice: /Stage[main]/Wazuh::Kibana_od/Exec[Verify Kibana folders owner]/returns: chown: cannot access ‘/usr/share/kibana/optimize’: No such file or directory
Error: 'chown -R kibana:kibana /usr/share/kibana/optimize && chown -R kibana:kibana /usr/share/kibana/plugins' returned 1 instead of one of [0]
Error: /Stage[main]/Wazuh::Kibana_od/Exec[Verify Kibana folders owner]/returns: change from 'notrun' to ['0'] failed: 'chown -R kibana:kibana /usr/share/kibana/optimize && chown -R kibana:kibana /usr/share/kibana/plugins' returned 1 instead of one of [0]
Info: Class[Wazuh::Kibana_od]: Unscheduling all events on Class[Wazuh::Kibana_od]
Notice: /Stage[main]/Wazuh::Repo_opendistro/Exec[Install Open Distro for Elasticsearch GPG key]/returns: executed successfully (corrective)
Error: Could not update: Execution of '/bin/yum -d 0 -e 0 -y install opendistroforelasticsearch-1.13.2' returned 1: Error: Package: opendistro-job-scheduler-1.13.0.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-sql-1.13.2.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-performance-analyzer-1.13.0.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-knn-1.13.0.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-security-1.13.1.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-alerting-1.13.1.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-anomaly-detection-1.13.0.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistroforelasticsearch-1.13.2-1.x86_64 (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-index-management-1.13.2.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-asynchronous-search-1.13.0.1-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-reports-scheduler-1.13.0.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
Error: /Stage[main]/Wazuh::Opendistro/Package[opendistroforelasticsearch]/ensure: change from 'purged' to '1.13.2' failed: Could not update: Execution of '/bin/yum -d 0 -e 0 -y install opendistroforelasticsearch-1.13.2' returned 1: Error: Package: opendistro-job-scheduler-1.13.0.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-sql-1.13.2.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-performance-analyzer-1.13.0.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-knn-1.13.0.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-security-1.13.1.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-alerting-1.13.1.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-anomaly-detection-1.13.0.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistroforelasticsearch-1.13.2-1.x86_64 (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-index-management-1.13.2.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-asynchronous-search-1.13.0.1-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
Error: Package: opendistro-reports-scheduler-1.13.0.0-1.noarch (opendistro)
Requires: elasticsearch-oss = 7.10.2
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
Notice: /Stage[main]/Wazuh::Opendistro/File[Configure elasticsearch.yml]: Dependency Package[opendistroforelasticsearch] has failures: true
Warning: /Stage[main]/Wazuh::Opendistro/File[Configure elasticsearch.yml]: Skipping because of failed dependencies
Warning: /Stage[main]/Wazuh::Opendistro/File[Configure jvm.options]: Skipping because of failed dependencies
Warning: /Stage[main]/Wazuh::Opendistro/Service[elasticsearch]: Skipping because of failed dependencies
Warning: /Stage[main]/Wazuh::Opendistro/Exec[Insert line limits]: Skipping because of failed dependencies
Warning: /Stage[main]/Wazuh::Opendistro/Exec[Verify Elasticsearch folders owner]: Skipping because of failed dependencies
Info: Stage[main]: Unscheduling all events on Stage[main]
Notice: Applied catalog in 359.99 secondsOn the Kibana od side, it is being reviewed because it cannot find the installation package, the path that is passed is correct, so it should not be generating a larger path where it is trying
On the Opendistro side, it is verifying which package needs to be installed, since if it is a dependency it should be installed
vcerenu
commented
2 years ago I was checking for Opendistro installation errors and it is fixed by installing filebeat first. Also, I am trying to access the repository where the wazuh-indexer packages are generated but they cannot be found, I have to continue verifying the correct configuration so that I can access them.
vcerenu
commented
2 years ago Fixed opendistro and wazuh-indexer install parameters causing install errors.
The repository in which the packages are being generated was configured, in order to carry out the installation tests.
An installation test was performed and it generated errors, possibly because the package found in the repository was a preliminary version and generated the following errors:
[root@agent ~]# puppet agent -t
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for agent
Info: Applying configuration version '1642186583'
Notice: /Stage[main]/Wazuh::Wazuh_indexer/Package[wazuh-indexer]/ensure: created
Notice: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure opensearch.yml]/content:
--- /etc/wazuh-indexer/opensearch.yml 2022-01-14 11:00:41.000000000 +0000
+++ /tmp/puppet-file20220114-24364-1inlfbn 2022-01-14 18:57:47.049862032 +0000
@@ -1,46 +1,89 @@
-network.host: "0.0.0.0"
-node.name: "node-1"
-cluster.initial_master_nodes:
-- "node-1"
-cluster.name: "wazuh-cluster"
-
-http.port: 9700-9799
-transport.tcp.port: 9800-9899
-node.max_local_storage_nodes: "3"
+# ======================== Elasticsearch Configuration =========================
+#
+# NOTE: Elasticsearch comes with reasonable defaults for most settings.
+# Before you set out to tweak and tune the configuration, make sure you
+# understand what are you trying to accomplish and the consequences.
+#
+# The primary way of configuring a node is via this file. This template lists
+# the most important settings you may want to configure for a production cluster.
+#
+# Please consult the documentation for further information on configuration options:
+# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
+#
+# ---------------------------------- Cluster -----------------------------------
+#
+# Use a descriptive name for your cluster:
+#
+cluster.name: es-wazuh
+#
+# ------------------------------------ Node ------------------------------------
+#
+# Use a descriptive name for the node:
+#
+node.name: node-01
+#
+# Add custom attributes to the node:
+#
+node.master: true
+#
+# ----------------------------------- Paths ------------------------------------
+#
+# Path to directory where to store the data (separate multiple locations by comma):
+#
path.data: /var/lib/wazuh-indexer
+#
+# Path to log files:
+#
path.logs: /var/log/wazuh-indexer
-
-
-###############################################################################
-# #
-# WARNING: Demo certificates set up in this file. #
-# Please change on production cluster! #
-# #
-###############################################################################
-
-plugins.security.ssl.http.pemcert_filepath: /etc/wazuh-indexer/certs/demo-indexer.pem
-plugins.security.ssl.http.pemkey_filepath: /etc/wazuh-indexer/certs/demo-indexer-key.pem
-plugins.security.ssl.http.pemtrustedcas_filepath: /etc/wazuh-indexer/certs/root-ca.pem
-plugins.security.ssl.transport.pemcert_filepath: /etc/wazuh-indexer/certs/demo-indexer.pem
-plugins.security.ssl.transport.pemkey_filepath: /etc/wazuh-indexer/certs/demo-indexer-key.pem
-plugins.security.ssl.transport.pemtrustedcas_filepath: /etc/wazuh-indexer/certs/root-ca.pem
-plugins.security.ssl.http.enabled: true
-plugins.security.ssl.transport.enforce_hostname_verification: false
-plugins.security.ssl.transport.resolve_hostname: false
-
-plugins.security.audit.type: internal_opensearch
-plugins.security.authcz.admin_dn:
-- "CN=admin,OU=Docu,O=Wazuh,L=California,C=US"
-plugins.security.check_snapshot_restore_write_privileges: true
-plugins.security.enable_snapshot_restore_privilege: true
-plugins.security.nodes_dn:
-- "CN=demo-indexer,OU=Docu,O=Wazuh,L=California,C=US"
-plugins.security.restapi.roles_enabled:
-- "all_access"
-- "security_rest_api_access"
-
-plugins.security.system_indices.enabled: true
-plugins.security.system_indices.indices: [".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opensearch-observability", ".opendistro-asynchronous-search-response*", ".replication-metadata-store"]
-
-### Option to allow Filebeat-oss 7.10.2 to work ###
-compatibility.override_main_response_version: true
\ No newline at end of file
+#
+# ----------------------------------- Memory -----------------------------------
+#
+# Lock the memory on startup:
+#
+#bootstrap.memory_lock: true
+#
+# Make sure that the heap size is set to about half the memory available
+# on the system and that the owner of the process is allowed to use this
+# limit.
+#
+# Elasticsearch performs poorly when the system is swapping the memory.
+#
+# ---------------------------------- Network -----------------------------------
+#
+# Set the bind address to a specific IP (IPv4 or IPv6):
+#
+network.host: localhost
+#
+# Set a custom port for HTTP:
+#
+http.port: 9700
+#
+# For more information, consult the network module documentation.
+#
+# --------------------------------- Discovery ----------------------------------
+#
+# Pass an initial list of hosts to perform discovery when this node is started:
+# The default list of hosts is ["127.0.0.1", "[::1]"]
+#
+#discovery.seed_hosts: ["host1", "host2"]
+#
+# Bootstrap the cluster using an initial set of master-eligible nodes:
+#
+#cluster.initial_master_nodes: ['node-01']
+discovery.type: single-node
+#
+# For more information, consult the discovery and cluster formation module documentation.
+#
+# ---------------------------------- Gateway -----------------------------------
+#
+# Block initial recovery after a full cluster restart until N nodes are started:
+#
+#gateway.recover_after_nodes: 3
+#
+# For more information, consult the gateway module documentation.
+#
+# ---------------------------------- Various -----------------------------------
+#
+# Require explicit names when deleting indices:
+#
+#action.destructive_requires_name: true
\ No newline at end of file
Info: Computing checksum on file /etc/wazuh-indexer/opensearch.yml
Info: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure opensearch.yml]: Filebucketed /etc/wazuh-indexer/opensearch.yml to puppet with sum 48a6755159c7730c29e560f1037000b9
Notice: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure opensearch.yml]/content: content changed '{md5}48a6755159c7730c29e560f1037000b9' to '{md5}5d23a8c135a6c203780773e4bbd27a6a'
Notice: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure opensearch.yml]/mode: mode changed '0660' to '0644'
Info: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure opensearch.yml]: Scheduling refresh of Service[wazuh-indexer]
Info: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure opensearch.yml]: Scheduling refresh of Service[wazuh-indexer]
Notice: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure disabledlog4j.options]/ensure: defined content as '{md5}7669fdd24c9fcb6898d6326e53e2a3cd'
Info: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure disabledlog4j.options]: Scheduling refresh of Service[wazuh-indexer]
Notice: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure jvm.options]/content:
--- /etc/wazuh-indexer/jvm.options 2021-12-28 12:09:28.000000000 +0000
+++ /tmp/puppet-file20220114-24364-1ca9yu6 2022-01-14 18:57:47.131862082 +0000
@@ -1,5 +1,3 @@
-## JVM configuration
-
################################################################
## IMPORTANT: JVM heap size
################################################################
@@ -11,7 +9,7 @@
## -Xms4g
## -Xmx4g
##
-## See https://opensearch.org/docs/opensearch/install/important-settings/
+## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################
@@ -38,17 +36,55 @@
8-13:-XX:+UseCMSInitiatingOccupancyOnly
## G1GC Configuration
-# NOTE: G1 GC is only supported on JDK version 10 or later
-# to use G1GC, uncomment the next two lines and update the version on the
-# following three lines to your version of the JDK
-# 10-13:-XX:-UseConcMarkSweepGC
-# 10-13:-XX:-UseCMSInitiatingOccupancyOnly
-14-:-XX:+UseG1GC
-14-:-XX:G1ReservePercent=25
-14-:-XX:InitiatingHeapOccupancyPercent=30
+# NOTE: G1GC is only supported on JDK version 10 or later.
+# To use G1GC uncomment the lines below.
+# 10-:-XX:-UseConcMarkSweepGC
+# 10-:-XX:-UseCMSInitiatingOccupancyOnly
+# 10-:-XX:+UseG1GC
+# 10-:-XX:InitiatingHeapOccupancyPercent=75
+
+## DNS cache policy
+# cache ttl in seconds for positive DNS lookups noting that this overrides the
+# JDK security property networkaddress.cache.ttl; set to -1 to cache forever
+-Des.networkaddress.cache.ttl=60
+# cache ttl in seconds for negative DNS lookups noting that this overrides the
+# JDK security property networkaddress.cache.negative ttl; set to -1 to cache
+# forever
+-Des.networkaddress.cache.negative.ttl=10
+
+## optimizations
+
+# pre-touch memory pages used by the JVM during initialization
+-XX:+AlwaysPreTouch
+
+## basic
+
+# explicitly set the stack size
+-Xss1m
+
+# set to headless, just in case
+-Djava.awt.headless=true
+
+# ensure UTF-8 encoding by default (e.g. filenames)
+-Dfile.encoding=UTF-8
+
+# use our provided JNA always versus the system one
+-Djna.nosys=true
+
+# turn off a JDK optimization that throws away stack traces for common
+# exceptions because stack traces are important for debugging
+-XX:-OmitStackTraceInFastThrow
+
+# flags to configure Netty
+-Dio.netty.noUnsafe=true
+-Dio.netty.noKeySetOptimization=true
+-Dio.netty.recycler.maxCapacityPerThread=0
+
+# log4j 2
+-Dlog4j.shutdownHookEnabled=false
+-Dlog4j2.disable.jmx=true
-## JVM temporary directory
--Djava.io.tmpdir=${OPENSEARCH_TMPDIR}
+-Djava.io.tmpdir=${ES_TMPDIR}
## heap dumps
@@ -58,21 +94,27 @@
# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
--XX:HeapDumpPath=data
+-XX:HeapDumpPath=/var/lib/elasticsearch
# specify an alternative path for JVM fatal error logs
--XX:ErrorFile=/var/log/wazuh-indexer/hs_err_pid%p.log
+-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log
## JDK 8 GC logging
+
8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
-8:-Xloggc:/var/log/wazuh-indexer/gc.log
+8:-Xloggc:/var/log/elasticsearch/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m
# JDK 9+ GC logging
-9-:-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/wazuh-indexer/gc.log:utctime,pid,tags:filecount=32,filesize=64m
+9-:-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m
+# due to internationalization enhancements in JDK 9 Elasticsearch need to set the provider to COMPAT otherwise
+# time/date parsing will break in an incompatible way for some date patterns and locals
+9-:-Djava.locale.providers=COMPAT
+# temporary workaround for C2 bug with JDK 10 on hardware with AVX-512
+10-:-XX:UseAVX=2
\ No newline at end of file
Info: Computing checksum on file /etc/wazuh-indexer/jvm.options
Info: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure jvm.options]: Filebucketed /etc/wazuh-indexer/jvm.options to puppet with sum 2348259ed7efed14bf76df5e01435e33
Notice: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure jvm.options]/content: content changed '{md5}2348259ed7efed14bf76df5e01435e33' to '{md5}b9f28d5dce5aaac484a606df44086e20'
Info: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure jvm.options]: Scheduling refresh of Service[wazuh-indexer]
Error: Systemd start for wazuh-indexer failed!
journalctl log for wazuh-indexer:
-- Logs begin at Fri 2022-01-14 15:42:46 UTC, end at Fri 2022-01-14 18:57:49 UTC. --
Jan 14 18:57:47 agent systemd[1]: Starting Wazuh-indexer...
Jan 14 18:57:49 agent systemd-entrypoint[24713]: Exception in thread "main" java.lang.RuntimeException: starting java failed with [1]
Jan 14 18:57:49 agent systemd-entrypoint[24713]: output:
Jan 14 18:57:49 agent systemd-entrypoint[24713]: [0.001s][error][logging] Error opening log file '/var/log/elasticsearch/gc.log': No such file or directory
Jan 14 18:57:49 agent systemd-entrypoint[24713]: [0.001s][error][logging] Initialization of output 'file=/var/log/elasticsearch/gc.log' using options 'filecount=32,filesize=64m' failed.
Jan 14 18:57:49 agent systemd-entrypoint[24713]: error:
Jan 14 18:57:49 agent systemd-entrypoint[24713]: Invalid -Xlog option '-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m', see error log for details.
Jan 14 18:57:49 agent systemd-entrypoint[24713]: Error: Could not create the Java Virtual Machine.
Jan 14 18:57:49 agent systemd-entrypoint[24713]: Error: A fatal exception has occurred. Program will exit.
Jan 14 18:57:49 agent systemd-entrypoint[24713]: at org.opensearch.tools.launchers.JvmErgonomics.flagsFinal(JvmErgonomics.java:139)
Jan 14 18:57:49 agent systemd-entrypoint[24713]: at org.opensearch.tools.launchers.JvmErgonomics.finalJvmOptions(JvmErgonomics.java:101)
Jan 14 18:57:49 agent systemd-entrypoint[24713]: at org.opensearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:72)
Jan 14 18:57:49 agent systemd-entrypoint[24713]: at org.opensearch.tools.launchers.JvmOptionsParser.jvmOptions(JvmOptionsParser.java:152)
Jan 14 18:57:49 agent systemd-entrypoint[24713]: at org.opensearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:110)
Jan 14 18:57:49 agent systemd[1]: wazuh-indexer.service: main process exited, code=exited, status=1/FAILURE
Jan 14 18:57:49 agent systemd[1]: Failed to start Wazuh-indexer.
Jan 14 18:57:49 agent systemd[1]: Unit wazuh-indexer.service entered failed state.
Jan 14 18:57:49 agent systemd[1]: wazuh-indexer.service failed.
Error: /Stage[main]/Wazuh::Wazuh_indexer/Service[wazuh-indexer]/ensure: change from 'stopped' to 'running' failed: Systemd start for wazuh-indexer failed!
journalctl log for wazuh-indexer:
-- Logs begin at Fri 2022-01-14 15:42:46 UTC, end at Fri 2022-01-14 18:57:49 UTC. --
Jan 14 18:57:47 agent systemd[1]: Starting Wazuh-indexer...
Jan 14 18:57:49 agent systemd-entrypoint[24713]: Exception in thread "main" java.lang.RuntimeException: starting java failed with [1]
Jan 14 18:57:49 agent systemd-entrypoint[24713]: output:
Jan 14 18:57:49 agent systemd-entrypoint[24713]: [0.001s][error][logging] Error opening log file '/var/log/elasticsearch/gc.log': No such file or directory
Jan 14 18:57:49 agent systemd-entrypoint[24713]: [0.001s][error][logging] Initialization of output 'file=/var/log/elasticsearch/gc.log' using options 'filecount=32,filesize=64m' failed.
Jan 14 18:57:49 agent systemd-entrypoint[24713]: error:
Jan 14 18:57:49 agent systemd-entrypoint[24713]: Invalid -Xlog option '-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m', see error log for details.
Jan 14 18:57:49 agent systemd-entrypoint[24713]: Error: Could not create the Java Virtual Machine.
Jan 14 18:57:49 agent systemd-entrypoint[24713]: Error: A fatal exception has occurred. Program will exit.
Jan 14 18:57:49 agent systemd-entrypoint[24713]: at org.opensearch.tools.launchers.JvmErgonomics.flagsFinal(JvmErgonomics.java:139)
Jan 14 18:57:49 agent systemd-entrypoint[24713]: at org.opensearch.tools.launchers.JvmErgonomics.finalJvmOptions(JvmErgonomics.java:101)
Jan 14 18:57:49 agent systemd-entrypoint[24713]: at org.opensearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:72)
Jan 14 18:57:49 agent systemd-entrypoint[24713]: at org.opensearch.tools.launchers.JvmOptionsParser.jvmOptions(JvmOptionsParser.java:152)
Jan 14 18:57:49 agent systemd-entrypoint[24713]: at org.opensearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:110)
Jan 14 18:57:49 agent systemd[1]: wazuh-indexer.service: main process exited, code=exited, status=1/FAILURE
Jan 14 18:57:49 agent systemd[1]: Failed to start Wazuh-indexer.
Jan 14 18:57:49 agent systemd[1]: Unit wazuh-indexer.service entered failed state.
Jan 14 18:57:49 agent systemd[1]: wazuh-indexer.service failed.
Notice: /Stage[main]/Wazuh::Wazuh_indexer/Service[wazuh-indexer]: Triggered 'refresh' from 4 events
Notice: /Stage[main]/Wazuh::Wazuh_indexer/Exec[Insert line limits]/returns: executed successfully
Notice: /Stage[main]/Wazuh::Wazuh_indexer/Exec[Verify wazuh-indexer folders owner]/returns: executed successfully
Info: Class[Wazuh::Wazuh_indexer]: Unscheduling all events on Class[Wazuh::Wazuh_indexer]
Notice: Applied catalog in 86.15 seconds
[root@agent ~]#It was able to generate the packages with the latest version of the wazuh-indexer code to be able to generate a new installation test, but it generates a new error, possibly because it cannot access the new generated package
[root@agent ~]# puppet agent -t
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for agent
Info: Applying configuration version '1642191949'
Error: Could not update: Execution of '/bin/yum -d 0 -e 0 -y install wazuh-indexer-4.3.0-0.0.0.todelete' returned 1: Error downloading packages:
wazuh-indexer-4.3.0-0.0.0.todelete.x86_64: [Errno 256] No more mirrors to try.
Error: /Stage[main]/Wazuh::Wazuh_indexer/Package[wazuh-indexer]/ensure: change from 'purged' to '4.3.0-0.0.0.todelete' failed: Could not update: Execution of '/bin/yum -d 0 -e 0 -y install wazuh-indexer-4.3.0-0.0.0.todelete' returned 1: Error downloading packages:
wazuh-indexer-4.3.0-0.0.0.todelete.x86_64: [Errno 256] No more mirrors to try. (corrective)
Notice: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure opensearch.yml]: Dependency Package[wazuh-indexer] has failures: true
Warning: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure opensearch.yml]: Skipping because of failed dependencies
Warning: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure disabledlog4j.options]: Skipping because of failed dependencies
Warning: /Stage[main]/Wazuh::Wazuh_indexer/File[Configure jvm.options]: Skipping because of failed dependencies
Warning: /Stage[main]/Wazuh::Wazuh_indexer/Service[wazuh-indexer]: Skipping because of failed dependencies
Warning: /Stage[main]/Wazuh::Wazuh_indexer/Exec[Insert line limits]: Skipping because of failed dependencies
Warning: /Stage[main]/Wazuh::Wazuh_indexer/Exec[Verify wazuh-indexer folders owner]: Skipping because of failed dependencies
Notice: Applied catalog in 17.39 seconds
[root@agent ~]#Added repository configuration to run new wazuh-indexer and wazuh-dashboard packages.
The manifests were modified with the necessary configurations so that both applications deploy without errors.
It is necessary to carry out an installation test together with wazuh-manager, to verify that wazuh-dashboard connects to the manager without errors, in addition to verifying the repository configuration, since it does not allow configuring repo in all manifests and when deploying more than one manifest at the same time it generates errors.
Deployment log:
[root@agent ~]# puppet agent -t
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for agent
Info: Applying configuration version '1642450393'
Notice: /Stage[main]/Wazuh::Repo/Yumrepo[wazuh]/ensure: created
Info: Yumrepo[wazuh](provider=inifile): changing mode of /etc/yum.repos.d/wazuh.repo from 600 to 644
Notice: /Stage[main]/Wazuh::Wazuh_indexer/Package[wazuh-indexer]/ensure: created
Notice: /Stage[main]/Wazuh::Wazuh_indexer/Service[wazuh-indexer]/ensure: ensure changed 'stopped' to 'running'
Info: /Stage[main]/Wazuh::Wazuh_indexer/Service[wazuh-indexer]: Unscheduling refresh on Service[wazuh-indexer]
Notice: /Stage[main]/Wazuh::Wazuh_indexer/Exec[Insert line limits]/returns: executed successfully
Notice: /Stage[main]/Wazuh::Wazuh_indexer/Exec[Verify wazuh-indexer folders owner]/returns: executed successfully
Notice: /Stage[main]/Wazuh::Wazuh_dashboard/Package[Installing Wazuh-dashboard...]/ensure: created
Notice: /Stage[main]/Wazuh::Wazuh_dashboard/Service[wazuh-dashboard]/ensure: ensure changed 'stopped' to 'running'
Info: /Stage[main]/Wazuh::Wazuh_dashboard/Service[wazuh-dashboard]: Unscheduling refresh on Service[wazuh-dashboard]
Notice: /Stage[main]/Wazuh::Wazuh_dashboard/Exec[Waiting for Wazuh indexer...]/returns: executed successfully (corrective)
Notice: Applied catalog in 155.57 seconds
[root@agent ~]# systemctl status wazuh-indexer
● wazuh-indexer.service - Wazuh-indexer
Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2022-01-17 20:14:53 UTC; 12min ago
Docs: https://documentation.wazuh.com
Main PID: 5012 (java)
CGroup: /system.slice/wazuh-indexer.service
└─5012 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 ...
Jan 17 20:14:31 agent systemd[1]: Starting Wazuh-indexer...
Jan 17 20:14:48 agent systemd-entrypoint[5012]: WARNING: An illegal reflective access operation has occurred
Jan 17 20:14:48 agent systemd-entrypoint[5012]: WARNING: Illegal reflective access by io.protostuff.runtime.PolymorphicThrowableSchema (file:/usr/share/wazuh-indexer/plugins/opensearch-anomaly-detection/protostuff-runt....Throwable.cause
Jan 17 20:14:48 agent systemd-entrypoint[5012]: WARNING: Please consider reporting this to the maintainers of io.protostuff.runtime.PolymorphicThrowableSchema
Jan 17 20:14:48 agent systemd-entrypoint[5012]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
Jan 17 20:14:48 agent systemd-entrypoint[5012]: WARNING: All illegal access operations will be denied in a future release
Jan 17 20:14:53 agent systemd[1]: Started Wazuh-indexer.
Hint: Some lines were ellipsized, use -l to show in full.
[root@agent ~]# systemctl status wazuh-dashboard
● wazuh-dashboard.service - wazuh-dashboard
Loaded: loaded (/etc/systemd/system/wazuh-dashboard.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2022-01-17 20:15:48 UTC; 11min ago
Main PID: 5305 (node)
CGroup: /system.slice/wazuh-dashboard.service
└─5305 /usr/share/wazuh-dashboard/bin/../node/bin/node /usr/share/wazuh-dashboard/bin/../src/cli/dist -c /etc/wazuh-dashboard/wazuh-dashboard.yml
Jan 17 20:27:14 agent opensearch-dashboards[5305]: {"type":"log","@timestamp":"2022-01-17T20:27:14Z","tags":["error","opensearch","data"],"pid":5305,"message":"[ResponseError]: Response Error"}
Jan 17 20:27:16 agent opensearch-dashboards[5305]: {"type":"log","@timestamp":"2022-01-17T20:27:16Z","tags":["error","opensearch","data"],"pid":5305,"message":"[ResponseError]: Response Error"}
Jan 17 20:27:19 agent opensearch-dashboards[5305]: {"type":"log","@timestamp":"2022-01-17T20:27:19Z","tags":["error","opensearch","data"],"pid":5305,"message":"[ResponseError]: Response Error"}
Jan 17 20:27:21 agent opensearch-dashboards[5305]: {"type":"log","@timestamp":"2022-01-17T20:27:21Z","tags":["error","opensearch","data"],"pid":5305,"message":"[ResponseError]: Response Error"}
Jan 17 20:27:24 agent opensearch-dashboards[5305]: {"type":"log","@timestamp":"2022-01-17T20:27:24Z","tags":["error","opensearch","data"],"pid":5305,"message":"[ResponseError]: Response Error"}
Jan 17 20:27:26 agent opensearch-dashboards[5305]: {"type":"log","@timestamp":"2022-01-17T20:27:26Z","tags":["error","opensearch","data"],"pid":5305,"message":"[ResponseError]: Response Error"}
Jan 17 20:27:29 agent opensearch-dashboards[5305]: {"type":"log","@timestamp":"2022-01-17T20:27:29Z","tags":["error","opensearch","data"],"pid":5305,"message":"[ResponseError]: Response Error"}
Jan 17 20:27:31 agent opensearch-dashboards[5305]: {"type":"log","@timestamp":"2022-01-17T20:27:31Z","tags":["error","opensearch","data"],"pid":5305,"message":"[ResponseError]: Response Error"}
Jan 17 20:27:34 agent opensearch-dashboards[5305]: {"type":"log","@timestamp":"2022-01-17T20:27:34Z","tags":["error","opensearch","data"],"pid":5305,"message":"[ResponseError]: Response Error"}
Jan 17 20:27:36 agent opensearch-dashboards[5305]: {"type":"log","@timestamp":"2022-01-17T20:27:36Z","tags":["error","opensearch","data"],"pid":5305,"message":"[ResponseError]: Response Error"}
[root@agent ~]#
alberpilot
commented
2 years ago
Analysis and creation of manifests for the installation of Wazuh Indexer with the puppet module