search

Yelp / elastalert

Easy & Flexible Alerting With ElasticSearch
https://elastalert.readthedocs.org
Apache License 2.0
7.99k stars 1.74k forks source link

Has match but not alert？ #2792

Open chisijun opened 4 years ago

chisijun commented 4 years ago

hello， i config rule match hits but not email alert。 WARNING:apscheduler.scheduler:Execution of job "ElastAlerter.handle_pending_alerts (trigger: interval[0:01:00], next run at: 2020-05-09 14:41:35 CST)" skipped: maximum number of running instances reached (1)

chisijun commented 4 years ago

INFO:elastalert:Background configuration change check run at 2020-05-09 14:38 CST INFO:elastalert:Disabled rules are: [] INFO:elastalert:Sleeping for 59.999853 seconds INFO:elastalert:Queried rule Nginx_err from 2020-05-09 14:33 CST to 2020-05-09 14:38 CST: 2 / 2 hits INFO:elastalert:Adding alert for Nginx_err to aggregation(id: VWkn-HEByoHdF7eIfW3m, aggregation_key: None), next alert at 2020-05-09 06:37:59.925795+00:00 INFO:elastalert:Ran Nginx_err from 2020-05-09 14:33 CST to 2020-05-09 14:38 CST: 2 query hits (1 already seen), 1 matches, 0 alerts sent WARNING:apscheduler.scheduler:Execution of job "ElastAlerter.handle_pending_alerts (trigger: interval[0:01:00], next run at: 2020-05-09 14:39:35 CST)" skipped: maximum number of running instances reached (1) INFO:elastalert:Background configuration change check run at 2020-05-09 14:39 CST INFO:elastalert:Disabled rules are: [] INFO:elastalert:Sleeping for 59.99985 seconds INFO:elastalert:Queried rule Nginx_err from 2020-05-09 14:33 CST to 2020-05-09 14:39 CST: 2 / 2 hits INFO:elastalert:Ran Nginx_err from 2020-05-09 14:33 CST to 2020-05-09 14:39 CST: 2 query hits (2 already seen), 0 matches, 0 alerts sent

rehannali commented 4 years ago

@chisijun , Hi, Could you provide us more details so we can see what's wrong? It didn't mentioned that it matched with query.

jwh5566 commented 4 years ago

+1 has match but not send email

INFO:elastalert:Sleeping for 119.999933 seconds INFO:elastalert:Queried rule Example frequency rule from 2020-05-27 15:02 CST to 2020-05-27 15:17 CST: 73 / 73 hits INFO:elastalert:Queried rule Example frequency rule from 2020-05-27 15:17 CST to 2020-05-27 15:32 CST: 144 / 144 hits INFO:elastalert:Queried rule Example frequency rule from 2020-05-27 15:32 CST to 2020-05-27 15:47 CST: 84 / 84 hits INFO:elastalert:Queried rule Example frequency rule from 2020-05-27 15:47 CST to 2020-05-27 16:02 CST: 31 / 31 hits INFO:elastalert:Queried rule Example frequency rule from 2020-05-27 16:02 CST to 2020-05-27 16:17 CST: 104 / 104 hits INFO:elastalert:Queried rule Example frequency rule from 2020-05-27 16:17 CST to 2020-05-27 16:32 CST: 50 / 50 hits INFO:elastalert:Queried rule Example frequency rule from 2020-05-27 16:32 CST to 2020-05-27 16:41 CST: 27 / 27 hits INFO:elastalert:Background configuration change check run at 2020-05-27 16:43 CST INFO:elastalert:Background alerts thread 0 pending alerts sent at 2020-05-27 16:43 CST INFO:elastalert:Disabled rules are: [] INFO:elastalert:Sleeping for 119.999904 seconds WARNING:apscheduler.scheduler:Execution of job "ElastAlerter.handle_rule_execution (trigger: interval[0:02:00], next run at: 2020-05-27 16:43:55 CST)" skipped: maximum number of running instances reached (1)

yinhejianke commented 4 years ago

WARNING:apscheduler.scheduler:Execution of job "ElastAlerter.handle_rule_execution (trigger: interval[0:01:00], next run at: 2020-09-02 07:52:53 UTC)" skipped: maximum number of running instances reached (1)

need help ！！！！

mengsir99 commented 4 years ago

WARNING:apscheduler.scheduler:Execution of job "ElastAlerter.handle_rule_execution (trigger: interval[0:01:00], next run at: 2020-09-02 07:52:53 UTC)" skipped: maximum number of running instances reached (1)

need help ！！！！

Have you solved it，I also met

yinhejianke commented 4 years ago

Try The Email send OK ? I find my email not send info .

ramprasadavirineni commented 3 years ago

Hi,

Could you please provide any fixes on below issue....I am not getting emails.

INFO:elastalert:Disabled rules are: [] INFO:elastalert:Sleeping for 59.999718 seconds INFO:elastalert:Queried rule Hello Test mail from ELK Stack please ignore from 2021-04-23 08:04 EDT to 2021-04-23 08:19 EDT: 0 / 0 hits INFO:elastalert:Ran Hello Test mail from ELK Stack please ignore from 2021-04-23 08:04 EDT to 2021-04-23 08:19 EDT: 0 query hits (0 already seen), 0 matches, 0 alerts sent ^CINFO:elastalert:SIGINT received, stopping ElastAlert.. ===========================================
Package Version

APScheduler 3.7.0 attrs 20.3.0 aws-requests-auth 0.4.3 blist 1.3.6 boto3 1.17.48 botocore 1.20.48 certifi 2020.12.5 cffi 1.14.5 chardet 4.0.0 configparser 5.0.2 croniter 1.0.11 cryptography 3.4.7 defusedxml 0.7.1 docopt 0.6.2 elastalert 0.2.4 elasticsearch 7.12.0 envparse 0.2.0 exotel 0.1.5 idna 2.10 importlib-metadata 3.10.0 jeepney 0.6.0 jira 3.0a2 jmespath 0.10.0 jsonschema 3.2.0 keyring 23.0.1 mock 4.0.3 oauthlib 3.1.0 pip 21.0.1 prison 0.1.3 pycparser 2.20 PyJWT 2.0.1 pyrsistent 0.17.3 PySocks 1.7.1 PyStaticConfiguration 0.10.5 python-dateutil 2.6.1 pytz 2021.1 PyYAML 5.4.1 requests 2.25.1 requests-oauthlib 1.3.0 requests-toolbelt 0.9.1 s3transfer 0.3.6 SecretStorage 3.3.1 setuptools 56.0.0 six 1.15.0 stomp.py 7.0.0 texttable 1.6.3 twilio 6.0.0 typing-extensions 3.7.4.3 tzlocal 2.1 urllib3 1.26.4 zipp 3.4.1

This is the folder that contains the rule yaml files

Any .yaml file will be loaded as a rule

rules_folder: example_rules

How often ElastAlert will query Elasticsearch

The unit can be anything from weeks to seconds

run_every: minutes: 1

ElastAlert will buffer results from the most recent

period of time, in case some log sources are not in real time

buffer_time: minutes: 15

The Elasticsearch hostname for metadata writeback

Note that every rule can have its own Elasticsearch host

es_host: xyz@example.com

The Elasticsearch port

es_port: 9200

The AWS region to use. Set this when using AWS-managed elasticsearch

aws_region: us-east-1

The AWS profile to use. Use this if you are using an aws-cli profile.

See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html

for details

profile: test

Optional URL prefix for Elasticsearch

es_url_prefix: elasticsearch

Connect with TLS to Elasticsearch

use_ssl: True

Verify TLS certificates

verify_certs: True

GET request with body is the default option for Elasticsearch.

If it fails for some reason, you can pass 'GET', 'POST' or 'source'.

See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport

for details

es_send_get_body_as: GET

Option basic-auth username and password for Elasticsearch

es_username: someusername

es_password: somepassword

Use SSL authentication with client certificates client_cert must be

a pem file containing both cert and key for client

verify_certs: True

ca_certs: /path/to/cacert.pem

client_cert: /path/to/client_cert.pem

client_key: /path/to/client_key.key

The index on es_host which is used for metadata storage

This can be a unmapped index, but it is recommended that you run

elastalert-create-index to set a mapping

writeback_index: elastalert_status writeback_alias: elastalert_alerts

If an alert fails for some reason, ElastAlert will retry

sending the alert until this time period has elapsed

alert_time_limit: days: 2

Custom logging configuration

If you want to setup your own logging configuration to log into

files as well or to Logstash and/or modify log levels, use

the configuration below and adjust to your needs.

Note: if you run ElastAlert with --verbose/--debug, the log level of

the "elastalert" logger is changed to INFO, if not already INFO/DEBUG.

logging:

version: 1

incremental: false

disable_existing_loggers: false

formatters:

logline:

format: '%(asctime)s %(levelname)+8s %(name)+20s %(message)s'

#

handlers:

console:

class: logging.StreamHandler

formatter: logline

level: DEBUG

stream: ext://sys.stderr

#

file:

class : logging.FileHandler

formatter: logline

level: DEBUG

filename: elastalert.log

#

loggers:

elastalert:

level: WARN

handlers: []

propagate: true

#

elasticsearch:

level: WARN

handlers: []

propagate: true

#

elasticsearch.trace:

level: WARN

handlers: []

propagate: true

#

'': # root logger

level: WARN

handlers:

- console

- file

propagate: false

===========================================================

Alert when the rate of events exceeds a threshold

(Optional)

Elasticsearch host

es_host: elasticsearch.example.com

(Optional)

Elasticsearch port

es_port: 14900

(OptionaL) Connect with SSL to Elasticsearch

use_ssl: True

(Optional) basic-auth username and password for Elasticsearch

es_username: someusername

es_password: somepassword

(Required)

Rule name, must be unique

name: Hello Test mail from ELK Stack please ignore

(Required)

Type of alert.

the frequency rule type alerts when num_events events occur with timeframe time

type: frequency

(Required)

Index to search, wildcard supported

index: filebeat-*

(Required, frequency specific)

Alert when this many documents matching the query occur within a timeframe

num_events: 3

(Required, frequency specific)

num_events must occur within this amount of time to trigger an alert

timeframe: hours: 1

(Required)

A list of Elasticsearch filters used for find events

These filters are joined with AND and nested in a filtered query

For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html

filter:

(Required)

The alert is use when a match is found

alert:

(required, email specific)

a list of email addresses to send alerts to

email: