Closed javimed closed 1 year ago
Edit: Added error when attempting https configuration
The documentation related to the Splunk tools are in the Splunk documentation site, https://docs.splunk.com/Documentation. The documentation of the integrations we published assumes someone wanting to integrate Wazuh into Splunk knows to deploy and operate Splunk.
I believe the cases you're describing are all related to the deployment, configuration and operation of Splunk.
While following the Splunk integration guide the following issues were found.
Forwarding events with logstash
Splunk Cloud: Unable to find valid certification path to requested target
```[ec2-user@ip-172-31-31-248 ~]$ sudo cat /etc/logstash/conf.d/wazuh-splunk.conf input { opensearch { hosts => ["172.31.28.206:9200"] user => "${WAZUH_INDEXER_USERNAME}" password => "${WAZUH_INDEXER_PASSWORD}" index => "wazuh-alerts-4.x-*" ssl => true ca_file => "/etc/logstash/wazuh-indexer-certs/root-ca.pem" query => '{ "query": { "range": { "@timestamp": { "gt": "now-1m" } } } }' schedule => "* * * * *" } } output { http { format => "json" # format of forwarded logs http_method => "post" # HTTP method used to forward logs url => "https://pr....splunkcloud.com:8088/services/collector/raw" # endpoint to forward logs to headers => ["Authorization", "Splunk ${SPLUNK_AUTH}"] cacert => "/etc/logstash/splunk-certs/pr...-splunkcloud-com.pem" } } [ec2-user@ip-172-31-31-248 ~]$ ``` ``` [ec2-user@ip-172-31-31-248 ~]$ sudo systemctl stop logstash [ec2-user@ip-172-31-31-248 ~]$ sudo -E /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/wazuh-splunk.conf --path.settings /etc/logstash/ Using bundled JDK: /usr/share/logstash/jdk Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties [2023-09-11T13:44:23,246][INFO ][logstash.runner ] Log4j configuration path used is: /etc/logstash/log4j2.properties [2023-09-11T13:44:23,249][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"8.9.2", "jruby.version"=>"jruby 9.3.10.0 (2.6.8) 2023-02-01 107b2e6697 OpenJDK 64-Bit Server VM 17.0.8+7 on 17.0.8+7 +indy +jit [x86_64-linux]"} [2023-09-11T13:44:23,254][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED] [2023-09-11T13:44:23,526][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/sinatra-2.2.4/lib/sinatra/base.rb:938: warning: constant Tilt::Cache is deprecated [2023-09-11T13:44:24,484][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false} [2023-09-11T13:44:25,761][INFO ][org.reflections.Reflections] Reflections took 291 ms to scan 1 urls, producing 132 keys and 464 values [2023-09-11T13:44:26,434][INFO ][logstash.javapipeline ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise. [2023-09-11T13:44:26,498][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, "pipeline.sources"=>["/etc/logstash/conf.d/wazuh-splunk.conf"], :thread=>"#Splunk Enterprise: Connection reset
``` [ec2-user@ip-172-31-31-248 bin]$ sudo systemctl stop logstash [ec2-user@ip-172-31-31-248 bin]$ sudo cat /etc/logstash/conf.d/wazuh-splunk.conf input { opensearch { hosts => ["172.31.28.206:9200"] user => "${WAZUH_INDEXER_USERNAME}" password => "${WAZUH_INDEXER_PASSWORD}" index => "wazuh-alerts-4.x-*" ssl => true ca_file => "/etc/logstash/wazuh-indexer-certs/root-ca.pem" query => '{ "query": { "range": { "@timestamp": { "gt": "now-1m" } } } }' schedule => "* * * * *" } } output { http { format => "json" # format of forwarded logs http_method => "post" # HTTP method used to forward logs url => "http://ip-172-31-31-248.us-west-1.compute.internal:8088/services/collector/raw" # endpoint to forward logs to headers => ["Authorization", "Splunk ${SPLUNK_AUTH}"] } } ``` ``` [ec2-user@ip-172-31-31-248 bin]$ sudo systemctl stop logstash [ec2-user@ip-172-31-31-248 bin]$ sudo -E /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/wazuh-splunk.conf --path.settings /etc/logstash/ Using bundled JDK: /usr/share/logstash/jdk Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties [2023-09-11T18:46:50,348][INFO ][logstash.runner ] Log4j configuration path used is: /etc/logstash/log4j2.properties [2023-09-11T18:46:50,351][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"8.9.2", "jruby.version"=>"jruby 9.3.10.0 (2.6.8) 2023-02-01 107b2e6697 OpenJDK 64-Bit Server VM 17.0.8+7 on 17.0.8+7 +indy +jit [x86_64-linux]"} [2023-09-11T18:46:50,353][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED] [2023-09-11T18:46:50,654][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/sinatra-2.2.4/lib/sinatra/base.rb:938: warning: constant Tilt::Cache is deprecated [2023-09-11T18:46:51,563][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false} [2023-09-11T18:46:52,601][INFO ][org.reflections.Reflections] Reflections took 240 ms to scan 1 urls, producing 132 keys and 464 values [2023-09-11T18:46:53,865][INFO ][logstash.javapipeline ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise. [2023-09-11T18:46:54,007][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, "pipeline.sources"=>["/etc/logstash/conf.d/wazuh-splunk.conf"], :thread=>"#Splunk Enterprise: Attempt with https
``` [ec2-user@ip-172-31-31-248 ~]$ sudo cat /etc/logstash/conf.d/wazuh-splunk.conf input { opensearch { hosts => ["172.31.28.206:9200"] user => "${WAZUH_INDEXER_USERNAME}" password => "${WAZUH_INDEXER_PASSWORD}" index => "wazuh-alerts-4.x-*" ssl => true ca_file => "/etc/logstash/wazuh-indexer-certs/root-ca.pem" query => '{ "query": { "range": { "@timestamp": { "gt": "now-1m" } } } }' schedule => "* * * * *" } } output { http { format => "json" # format of forwarded logs http_method => "post" # HTTP method used to forward logs url => "https://172.31.31.248:8088/services/collector/raw" # endpoint to forward logs to headers => ["Authorization", "Splunk ${SPLUNK_AUTH}"] cacert => "/etc/logstash/splunk-certs/ca.pem" } } ``` ``` [ec2-user@ip-172-31-31-248 ~]$ sudo -E /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/wazuh-splunk.conf --path.settings /etc/logstash/ Using bundled JDK: /usr/share/logstash/jdk Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties [2023-09-12T19:20:28,997][INFO ][logstash.runner ] Log4j configuration path used is: /etc/logstash/log4j2.properties [2023-09-12T19:20:28,999][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"8.9.2", "jruby.version"=>"jruby 9.3.10.0 (2.6.8) 2023-02-01 107b2e6697 OpenJDK 64-Bit Server VM 17.0.8+7 on 17.0.8+7 +indy +jit [x86_64-linux]"} [2023-09-12T19:20:29,007][INFO ][logstash.runner ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED] [2023-09-12T19:20:29,632][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified /usr/share/logstash/vendor/bundle/jruby/2.6.0/gems/sinatra-2.2.4/lib/sinatra/base.rb:938: warning: constant Tilt::Cache is deprecated [2023-09-12T19:20:31,934][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false} [2023-09-12T19:20:33,306][INFO ][org.reflections.Reflections] Reflections took 248 ms to scan 1 urls, producing 132 keys and 464 values [2023-09-12T19:20:34,469][INFO ][logstash.javapipeline ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise. [2023-09-12T19:20:34,518][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, "pipeline.sources"=>["/etc/logstash/conf.d/wazuh-splunk.conf"], :thread=>"#Configuring the receiving port in Splunk Cloud: Missing Forwarding and receiving option
https://user-images.githubusercontent.com/47069802/266863359-6447cc4e-defc-4a9a-918a-d7476ff7251a.png
Installing Splunk Forwarder: SSL certificate generation failed