Loading Filters Directly From Kibana 3 not working

[audrjon]

Hi,

First, Thanks for a great project! But I have problems when I am following "Loading Filters Directly From Kibana 3" description. It seems, that it is not working properly. To test it, I created 3 the same rules, but they were generated using different options:

1.yaml (according to Loading Filters Directly From Kibana 3):

filter:
  download_dashboard: "Netflow"

2.yaml - generated by rule elastalert-rule-from-kibana: Generation procedure: Elasticsearch host: 127.0.0.1 Elasticsearch port: 9200 Dashboard name: Netflow Method for querying Elasticsearch[GET]:

Partial Config file
-----------
name: Netflow
es_host: 127.0.0.1
es_port: 9200
filter:
- term: {dst_ip: '"8.8.8.8"'}

---

2.yaml config:

filter:
- term: {dst_ip: '"8.8.8.8"'}

3.yaml - changed manualy, according to documentation: filter:

- term: 
    dst_ip: "8.8.8.8"

Only the third one is working. Why is that?

My Setup:

• elastalert - current release
• elasticsearch 1.7.5
• kibana3

[Qmando]

It seems to be doubling the quotation marks in the first two examples. I think this is because you have double quotes in your Kibana dashboard filter. Maybe those are being stripped out somewhere in Kibana which is why your dashboard works.

[Qmando]

Oh, I actually have the reverse https://github.com/Yelp/elastalert/blob/master/elastalert/kibana.py#L217

I guess ElastAlert should be stripping off quotation marks for some filter types. Sorry about that.

[audrjon]

Hi Qmando,

Yes, just checked, if you remove the quotes and leave jus '8.8.8.8', rule works as expected. So, you was right. Also I found some additional problems. In kibana3 you can create a filter in several ways. The most usual:

• first, when you filter in terms pane. So in that case you will get in filtering tab:

  terms must  
  field : dst_ip
  value : 8.8.8.8

• second, when you create a filter from table pane. In that case you will get:

  field must 
  field : dst_ip
  query : "8.8.8.8"

When you save a dashboard in first case and run the rule elastalert-rule-from-kibana, you get an error

elastalert-rule-from-kibana 
Elasticsearch host: 127.0.0.1
Elasticsearch port: 9200
Dashboard name: Netflow
Method for querying Elasticsearch[GET]: 
Traceback (most recent call last):
  File "/usr/local/bin/elastalert-rule-from-kibana", line 11, in <module>
    load_entry_point('elastalert==0.1.16', 'console_scripts', 'elastalert-rule-from-kibana')()
  File "/usr/local/lib/python2.7/dist-packages/elastalert-0.1.16-py2.7.egg/elastalert/rule_from_kibana.py", line 27, in main
    config_filters = filters_from_dashboard(db)
  File "/usr/local/lib/python2.7/dist-packages/elastalert-0.1.16-py2.7.egg/elastalert/kibana.py", line 273, in filters_from_dashboard
    config_filters.append(config_filter)
UnboundLocalError: local variable 'config_filter' referenced before assignment

When you save a dashboard in second case you get almost correct config file, but you need to remove double quotes:

elastalert-rule-from-kibana              
Elasticsearch host: 127.0.0.1
Elasticsearch port: 9200
Dashboard name: Netflow
Method for querying Elasticsearch[GET]: 

Partial Config file
-----------

name: Netflow
es_host: 127.0.0.1
es_port: 9200
filter:
- term: {dst_ip: '"8.8.8.8"'}

Also in kibana3 you can filter dashboard based on query and filter. But in elastalert just filter works. Query search is ignored. This is by design?