Could not generate Kibana dash using query_string filter

[zhangyuxuan1992]

rule config:

    # the frequency rule type alerts when num_events events occur with timeframe time
    type: frequency

    # (Required)
    # Index to search, wildcard supported
    index: logstash-celerx-*

    include:
      - _index

    # (Required, frequency specific)
    # Alert when this many documents matching the query occur within a timeframe
    num_events: 10

    # (Required, frequency specific)
    # num_events must occur within this amount of time to trigger an alert
    timeframe:
      minutes: 1

    kibana_url: "https://***/kibana/"
    generate_kibana_link: True

    filter:
    - query:
        query_string:
          query: "loglevel: ERROR"
    use_count_query: True
    doc_type: _doc
    query_key: _index

ERROR :

frequency_rules.yaml
Didn't get any results.
1 rules loaded
INFO:apscheduler.scheduler:Adding job tentatively -- it will be properly scheduled when the scheduler starts
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:30 UTC to 2020-10-16 04:31 UTC: 0 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:31 UTC to 2020-10-16 04:32 UTC: 0 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:32 UTC to 2020-10-16 04:33 UTC: 0 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:33 UTC to 2020-10-16 04:34 UTC: 0 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:34 UTC to 2020-10-16 04:35 UTC: 575 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:35 UTC to 2020-10-16 04:36 UTC: 0 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:36 UTC to 2020-10-16 04:37 UTC: 317 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:37 UTC to 2020-10-16 04:38 UTC: 1479 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:38 UTC to 2020-10-16 04:39 UTC: 1386 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:39 UTC to 2020-10-16 04:40 UTC: 0 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:40 UTC to 2020-10-16 04:41 UTC: 0 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:41 UTC to 2020-10-16 04:42 UTC: 0 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:42 UTC to 2020-10-16 04:43 UTC: 0 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:43 UTC to 2020-10-16 04:44 UTC: 0 hits
INFO:elastalert:Queried rule /opt/rules/frequency_rules from 2020-10-16 04:44 UTC to 2020-10-16 04:45 UTC: 0 hits
ERROR:root:Could not generate Kibana dash for /opt/rules/frequency_rules match: Could not parse filter {'query_string': {'query': 'loglevel: ERROR'}} for Kibana
/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:986: InsecureRequestWarning: Unverified HTTPS request is being made to host 'hooks.slack.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
INFO:elastalert:Alert '/opt/rules/frequency_rules' sent to Slack
ERROR:root:Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 1450, in alert
    return self.send_alert(matches, rule, alert_time=alert_time, retried=retried)
  File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 1549, in send_alert
    self.thread_data.alerts_sent += 1
AttributeError: '_thread._local' object has no attribute 'alerts_sent'

ERROR:root:Uncaught exception running rule /opt/rules/frequency_rules: '_thread._local' object has no attribute 'alerts_sent'
INFO:elastalert:Ignoring match for silenced rule /opt/rules/frequency_rules
INFO:elastalert:Ignoring match for silenced rule /opt/rules/frequency_rules
INFO:elastalert:Ignoring match for silenced rule /opt/rules/frequency_rules

Would have written the following documents to writeback index (default is elastalert_status):

silence - {'exponent': 0, 'rule_name': '/opt/rules/frequency_rules', '@timestamp': datetime.datetime(2020, 10, 19, 6, 7, 52, 821248, tzinfo=tzutc()), 'until': datetime.datetime(2020, 10, 19, 6, 8, 52, 821229, tzinfo=tzutc())}

elastalert_error - {'message': "Could not generate Kibana dash for /opt/rules/frequency_rules match: Could not parse filter {'query_string': {'query': 'loglevel: ERROR'}} for Kibana", 'traceback': ['Traceback (most recent call last):', '  File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 1492, in send_alert', '    kb_link = self.generate_kibana_db(rule, matches[0])', '  File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 1348, in generate_kibana_db', '    kibana.add_filter(db, filter)', '  File "/usr/local/lib/python3.6/site-packages/elastalert/kibana.py", line 238, in add_filter', '    raise EAException("Could not parse filter %s for Kibana" % (es_filter))', "elastalert.util.EAException: Could not parse filter {'query_string': {'query': 'loglevel: ERROR'}} for Kibana"]}

elastalert_error - {'message': "Uncaught exception running rule /opt/rules/frequency_rules: '_thread._local' object has no attribute 'alerts_sent'", 'traceback': ['Traceback (most recent call last):', '  File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 1450, in alert', '    return self.send_alert(matches, rule, alert_time=alert_time, retried=retried)', '  File "/usr/local/lib/python3.6/site-packages/elastalert/elastalert.py", line 1549, in send_alert', '    self.thread_data.alerts_sent += 1', "AttributeError: '_thread._local' object has no attribute 'alerts_sent'"], 'data': {'rule': '/opt/rules/frequency_rules'}}

elastalert_status - {'rule_name': '/opt/rules/frequency_rules', 'endtime': datetime.datetime(2020, 10, 16, 4, 45, tzinfo=<UTC>), 'starttime': datetime.datetime(2020, 10, 16, 4, 30, tzinfo=<UTC>), 'matches': 4, 'hits': 3757, '@timestamp': datetime.datetime(2020, 10, 19, 6, 7, 53, 100668, tzinfo=tzutc()), 'time_taken': 0.6728076934814453}

sys:1: ResourceWarning: unclosed <socket.socket fd=3, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('192.168.248.136', 53836), raddr=('10.100.240.74', 9200)>```

[zhangyuxuan1992]

look Issue in elastalert.py #2556

raise EAException("Could not parse filter %s for Kibana" % (es_filter))', "elastalert.util.EAException: Could not parse filter {'query_string': {'query': 'loglevel: ERROR'}} for Kibana"]}`

my question is this error

[zhangyuxuan1992]

so is there generate_kibana_link for es 7?

[venmaniselvan]

@nsano-rururu. Then how to generate_kibana_link for es 7. elastalert only support for es3 and es 4? If support es 7 means can you share an example here, please. Thank you did you solve this problem @zhangyuxuan1992 ?

[venmaniselvan]

Thank you @nsano-rururu Finally, I changed like this. It's working fine.

name: Example frequency rule
type: frequency
index: audit-log-*
num_events: 1
timeframe:
  minutes: 5

filter:
- query:
    query_string:
      query: "applicationname.keyword: SRTransform"
alert:
- "email"
email:
- "abcd@gmail.com"
from_addr: "no-reply@elastalert.com"
email_reply_to: "no-reply@elastalert.com"
aggregation:
  minutes: 2

alert_text_type: alert_text_only
alert_text: "Generated link to kibana: {0}"
alert_text_args: ["kibana_link"]
# Kibana
kibana_url: http://x.x.x.x:5601/app/kibana
use_kibana4_dashboard: "http://x.x.x.x:5601/app/kibana#/dashboard/78bce050-a193-11ea-ab49-e5346031d124"

NOTE: 78bce050-a193-11ea-ab49-e5346031d124 is Dashboard ID.