search

bitsensor / elastalert-kibana-plugin

ElastAlert Kibana Plugin
https://bitsensor.io/blog/elastalert-kibana-plugin-centralized-logging-with-integrated-alerting
Other
556 stars 118 forks source link

Required config for ssl config for elk #164

Open ck-7 opened 3 years ago

ck-7 commented 3 years ago

We configured the ELK with ssl based. How to config Elastalert with SSL based authentication? please share the SSL parameter to pass it config.json and elastalert.yaml ??

Kibana.yml

[root@elk-logging elastalert]# cat /etc/kibana/kibana.yml server.host: "elk-logging" server.port: 5601 elasticsearch.hosts: ["https://elk-logging.xxcxcx.net:9200"] elasticsearch.password: XXXXXCXCX

Elasticsearch from/to Kibana

elasticsearch.ssl.certificateAuthorities: /etc/kibana/certs/ca/ca.crt elasticsearch.ssl.certificate: /etc/kibana/certs/kibana.crt elasticsearch.ssl.key: /etc/kibana/certs/kibana.key elasticsearch.ssl.verificationMode: none

Browser from/to Kibana

server.ssl.enabled: true server.ssl.certificate: /etc/kibana/certs/kibana.crt server.ssl.key: /etc/kibana/certs/kibana.key

Elasticsearch authentication

xpack.security.enabled: true elasticsearch.username: elastic server.defaultRoute: /app/wazuh

Elastalert Hosts

elastalert-kibana-plugin.serverHost: elk-logging elastalert-kibana-plugin.serverPort: 3030

elastalert -- config [root@elk-logging config]# cat config.json { "appName": "elastalert-server", "port": 3030, "wsport": 3333, "elastalertPath": "/opt/elastalert", "verbose": true, "es_debug": true, "debug": true, "rulesPath": { "relative": true, "path": "/rules" }, "templatesPath": { "relative": true, "path": "/rule_templates" }, "es_host": "elk-logging", "es_username": "elastic", // Option basic-auth username and password for Elasticsearch "es_password": "XXXXXCXCX", // Option basic-auth username and password for Elasticsearch "es_ssl": true, // Enable/Disable SSL "es_ca_certs": "/etc/elasticsearch/certs/ca/ca.crt", // Path to ca for ElasticSearch (SSL must be enabled) "es_client_cert": "/etc/elasticsearch/certs/elasticsearch.crt", // Path to cert for ElasticSearch (SSL must be enabled) "es_client_key": "/etc/elasticsearch/certs/elasticsearch.key", // Path to key for ElasticSearch (SSL must be enabled) "es_port": 9200, "writeback_index": "elastalert_status" }

Elastalert Elasticsearch.yaml

The elasticsearch hostname for metadata writeback

Note that every rule can have its own elasticsearch host

es_host: elk-logging

The elasticsearch port

es_port: 9200

This is the folder that contains the rule yaml files

Any .yaml file will be loaded as a rule

rules_folder: rules

How often ElastAlert will query elasticsearch

The unit can be anything from weeks to seconds

run_every: seconds: 5

ElastAlert will buffer results from the most recent

period of time, in case some log sources are not in real time

buffer_time: minutes: 1

Optional URL prefix for elasticsearch

es_url_prefix: elasticsearch

Connect with TLS to elasticsearch

use_ssl: True

Verify TLS certificates

verify_certs: True client_cert: "/etc/elasticsearch/certs/elasticsearch.crt" client_key: "/etc/elasticsearch/certs/elasticsearch.key" ca_certs: "/etc/elasticsearch/certs/ca/ca.crt"

GET request with body is the default option for Elasticsearch.

If it fails for some reason, you can pass 'GET', 'POST' or 'source'.

See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport

for details

es_send_get_body_as: GET

Option basic-auth username and password for elasticsearch

es_username: elastic es_password: XXXXXXCXXX

The index on es_host which is used for metadata storage

This can be a unmapped index, but it is recommended that you run

elastalert-create-index to set a mapping

writeback_index: elastalert_status

If an alert fails for some reason, ElastAlert will retry

sending the alert until this time period has elapsed

alert_time_limit: days: 2

[root@elk-logging elastalert]# docker start --interactive elastalert

@bitsensor/elastalert@3.0.0-beta.0 start /opt/elastalert-server sh ./scripts/start.sh

14:31:12.693Z INFO elastalert-server: Config: No config.dev.json file was found in /opt/elastalert-server/config/config.dev.json. 14:31:12.696Z INFO elastalert-server: Config: Proceeding to look for normal config file. 14:31:12.697Z INFO elastalert-server: Config: A config file was found in /opt/elastalert-server/config/config.json. Using that config. 14:31:12.720Z INFO elastalert-server: Router: Listening for GET request on /. 14:31:12.720Z INFO elastalert-server: Router: Listening for GET request on /status. 14:31:12.721Z INFO elastalert-server: Router: Listening for GET request on /status/control/:action. 14:31:12.721Z INFO elastalert-server: Router: Listening for GET request on /status/errors. 14:31:12.721Z INFO elastalert-server: Router: Listening for GET request on /rules. 14:31:12.723Z INFO elastalert-server: Router: Listening for GET request on /rules/:id. 14:31:12.723Z INFO elastalert-server: Router: Listening for POST request on /rules/:id. 14:31:12.723Z INFO elastalert-server: Router: Listening for DELETE request on /rules/:id. 14:31:12.723Z INFO elastalert-server: Router: Listening for GET request on /templates. 14:31:12.723Z INFO elastalert-server: Router: Listening for GET request on /templates/:id. 14:31:12.724Z INFO elastalert-server: Router: Listening for POST request on /templates/:id. 14:31:12.724Z INFO elastalert-server: Router: Listening for DELETE request on /templates/:id. 14:31:12.724Z INFO elastalert-server: Router: Listening for POST request on /test. 14:31:12.724Z INFO elastalert-server: Router: Listening for GET request on /config. 14:31:12.724Z INFO elastalert-server: Router: Listening for POST request on /config. 14:31:12.724Z INFO elastalert-server: Router: Listening for POST request on /download. 14:31:12.724Z INFO elastalert-server: Router: Listening for GET request on /metadata/:type. 14:31:12.725Z INFO elastalert-server: Router: Listening for GET request on /mapping/:index. 14:31:12.725Z INFO elastalert-server: Router: Listening for POST request on /search/:index. 14:31:12.742Z ERROR elastalert-server: Server: Starting server failed with error: TypeError: object must be passed at module.exports (/opt/elastalert-server/node_modules/object-resolve-path/object-resolve-path.js:13:11) at ServerConfig.get (/opt/elastalert-server/src/common/config/server_config.js:32:12) at /opt/elastalert-server/src/elastalert_server.js:67:58 at /opt/elastalert-server/src/common/config/server_config.js:60:9 at Array.forEach () at /opt/elastalert-server/src/common/config/server_config.js:59:22 14:31:12.742Z INFO elastalert-server: Server: Stopping server /opt/elastalert-server/src/common/websocket.js:34 wss.clients.forEach(function (ws) { ^

TypeError: Cannot read property 'clients' of null at Timeout._onTimeout (/opt/elastalert-server/src/common/websocket.js:22:7) at listOnTimeout (internal/timers.js:531:17) at processTimers (internal/timer

nsano-rururu commented 3 years ago

kibana.yml configurations -

elastalert-kibana-plugin.serverHost: <HostName.Domain.com>
elastalert-kibana-plugin.serverPort: 443
elastalert-kibana-plugin.serverSsl: true
nsano-rururu commented 3 years ago

Is it docker? .. What is the docker image name of elastalert-server specified?

nsano-rururu commented 3 years ago

If you do not delete the comments after "//" and "//", an error should occur. Also, are the es_ca_certs, es_client_cert, and es_client_key files mounted when the docker container is started so that they can be referenced from within the docker container?

"es_username": "elastic", // Option basic-auth username and password for Elasticsearch
"es_password": "XXXXXCXCX", // Option basic-auth username and password for Elasticsearch
"es_ssl": true, // Enable/Disable SSL
"es_ca_certs": "/etc/elasticsearch/certs/ca/ca.crt", // Path to ca for ElasticSearch (SSL must be enabled)
"es_client_cert": "/etc/elasticsearch/certs/elasticsearch.crt", // Path to cert for ElasticSearch (SSL must be enabled)
"es_client_key": "/etc/elasticsearch/certs/elasticsearch.key", // Path to key for ElasticSearch (SSL must be enabled)

after

"es_username": "elastic",
"es_password": "XXXXXCXCX",
"es_ssl": true,
"es_ca_certs": "/etc/elasticsearch/certs/ca/ca.crt",
"es_client_cert": "/etc/elasticsearch/certs/elasticsearch.crt",
"es_client_key": "/etc/elasticsearch/certs/elasticsearch.key",
nsano-rururu commented 3 years ago

As a precaution when debugging, note that the alert will not be skipped if debug is set to true in config.json of ElastAlert Server.

example

  "es_debug": false,
  "debug": false,
nsano-rururu commented 3 years ago

I think that elastalert-kibana-plugin has been discontinued, so I don't think it will be fixed even if it doesn't work due to a bug.

ck-7 commented 3 years ago

Is it docker? .. What is the docker image name of elastalert-server specified?

with latest image bitsensor/elastalert:3.0.0-beta.0

ck-7 commented 3 years ago

I think that elastalert-kibana-plugin has been discontinued, so I don't think it will be fixed even if it doesn't work due to a bug.

any alternative tool for GUI Based Kibana alert Plugin?

nsano-rururu commented 3 years ago

with latest image bitsensor/elastalert:3.0.0-beta.0

bitsensor / elastalert does not have the following settings. This is a setting added to the fork johnsusek / elastalert-server (former repository ServerCentral / elastalert-server).

"es_ssl": true,
"es_ca_certs": "/etc/elasticsearch/certs/ca/ca.crt",
"es_client_cert": "/etc/elasticsearch/certs/elasticsearch.crt",
"es_client_key": "/etc/elasticsearch/certs/elasticsearch.key",
nsano-rururu commented 3 years ago

any alternative tool for GUI Based Kibana alert Plugin?

Praeco. By the way, I'm the co-maintainer of Praeco. https://github.com/johnsusek/praeco

nsano-rururu commented 3 years ago

By the way, I'm also the co-maintainer of johnsusek / elastalert-server. https://github.com/johnsusek/elastalert-server

nsano-rururu commented 3 years ago

It forks elastalert-kibana-plugin and supports up to Kibana 7.9.3. It needs to be remade to support Kibana 7.10 or later, and it has not been started yet. https://github.com/nsano-rururu/elastalert-kibana-plugin

ck-7 commented 3 years ago

It forks elastalert-kibana-plugin and supports up to Kibana 7.9.3. It needs to be remade to support Kibana 7.10 or later, and it has not been started yet. https://github.com/nsano-rururu/elastalert-kibana-plugin

by the way am using kibana 7.9.3 version for the elastalert kibana plugin..

ck-7 commented 3 years ago

It forks elastalert-kibana-plugin and supports up to Kibana 7.9.3. It needs to be remade to support Kibana 7.10 or later, and it has not been started yet. https://github.com/nsano-rururu/elastalert-kibana-plugin

Looking for it ASAP. Thank for the quick reply and response.

nsano-rururu commented 2 years ago

Ask a question in the repository you are maintaining https://github.com/Karql/elastalert-kibana-plugin