Kibana Dashboard filtering src_ip matches to many entries

[jahlives]

Hi I have the problem, that the filter in kibana dashboard does not work properly. I wanted to filter out connections made by t-pot for ubuntu updates. So I tried to set a filter like

query: src_ip.raw=192.168.254.2

and "policy" "must not" where 192.168.254.2 is the internal ip of my t-pot system. But that filters out ANY records even those which src_ip.raw is NOT 192.168.254.2 Is that a bug or am I not understanding the filters correctly?

[t3chn0m4g3]

@jahlives filtering the source ip might also filter packets you want to see. i had the best results with a destination filter. the following query filters destination ip matches with 1.1.1.1 or 2.2.2.2:

• querystring mustNot
• dest_ip="1.1.1.1" or dest_ip="2.2.2.2"

please let me know if that works for you, too.

[jahlives]

@t3chn0m4g3 I solved it with three more filters which I added to default.json

• must not http.hostname="security.ubuntu.com"
• must not http_hostname="archive.ubuntu.com"
• must not alert.signature="ET POLICY GNU/Linux APT User-Agent Outbound likely related to package management"

[t3chn0m4g3]

@jahlives looks good, will update default.json in next release, you can track here: https://github.com/dtag-dev-sec/elk/issues/2

[jahlives]

@t3chn0m4g3 I checked the filter rules on my sensor and saw one downsize of it: you should add the dst_ip to the filter with the ip address of the sensor system. Otherwise a bad guy could "attack" the sensor by adding the sensors ip address as archive|security.ubuntu.com In that case the filter rule would not show these "attacks" Solution: add as dst_ip the sensors ip address connected with a logical AND operator to every of the three rules. Meaning: filter out if security.ubuntu.com AND dst_ip != sensors_ip Unfortunately I could not add the dst_ip to the rules. These filters should only apply if the dst is not the sensors ip address

[t3chn0m4g3]

@jahlives thanks for the update!

[jahlives]

Is there any way to achieve it with a filter that security.ubuntu.com and archive.ubuntu.com is ignored BUT only if the dst_ip IS NOT the ip of the sensor system? I tried a lot but no success so far :-)

[t3chn0m4g3]

@jahlives the filter subset AFAIK has no option building such a filter (nice idea BTW). i guess there is no general solution possible that we could centrally implement. some sensors are behind network / port address translations, others change their ip on a daily basis, etc. that is the downside of using filters, you will always reduce the information. kibana however has the advantage that you can turn a filter on and off, so if something looks suspicious you can access all information easily.