Closed bennneuh closed 8 years ago
Qmando
commented
8 years ago It must be GREATER than 2, not equal to 2.
You can see what values exist at any given time if you do this:
At this line (https://github.com/Yelp/elastalert/blob/896985503f8978cfbc3d332888e46f2ae875a0e8/elastalert/ruletypes.py#L778) add print self.cardinality_cache[key].keys()
If you see three values, and then it still doesn't alert, then one of the items is more than 10 seconds old.
bennneuh
commented
8 years ago Yes i have tested greater than 2. But the rule hit, but don't match.
I will test the print because yes, i think i need to see cardinatilies fealds identified and the counters associated to be sure if it match correctly.
Thanks for the link. Le 9 juil. 2016 01:51, "Quentin Long" notifications@github.com a écrit :
It must be GREATER than 2, not equal to 2. You can see what values exist at any given time if you do this: At this line ( https://github.com/Yelp/elastalert/blob/896985503f8978cfbc3d332888e46f2ae875a0e8/elastalert/ruletypes.py#L778) add print self.cardinality_cache[key].keys()
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Yelp/elastalert/issues/624#issuecomment-231498693, or mute the thread https://github.com/notifications/unsubscribe/AH8WB1LZpUCd4KUNv0nMuDyNzmFCslAvks5qTuJ0gaJpZM4JIes2 .
bennneuh
commented
8 years ago Ok with some prints, i have understood. It's me, sorry, i have misunderstanding the cardinality rule... Very sorry for the time lost...
Do you know if it's possible to create rule to match my initial objective:
. If the rule match the filter . During the timeframe . I want to check all values of the cardinality field . And count each "iteration" of the same value (example : 2 logs with the usename "bob") . If the iteration is greater than a specific value i generate the alert
Example: Timeframe 2 minutes max cardinality (or equivalent): 2 Cardinality field : "sender_ip" During 2 minutes i have received 5 logs correctly filtered with sender_ip : 1.1.1.1, with my objective of rule, an alert is generated (greater than the "max_cardinality or equivalent" And during the same period i have received 2 logs correctly filtered with sender_ip : 2.2.2.2 who must not generate alert because it's less than the max_cardinality
I hope i'm clear.
Do i have to use the query_key ?
Thanks by advance,
Benoît
bennneuh
commented
8 years ago Or maybe i need to use the frequency rule with query_key ?
bennneuh
commented
8 years ago Ok i have tested frequency rule with query_key and yes it's the solution. thanks Quentin for your help! :)
sathishdsgithub
commented
7 years ago Cardinality query hits but no matches found !! I'm trying to use the query_key to match the source and destination IP address and the cardinality_field for the destination port. The logic of the query is if the max_cardinality is equal or above 50 (unique destination port) it should trigger an alert . But in my case the rule identifies the hits but does not produce the matches:-(
The below rule says query hits with 121 but does not produce the alert matches
root@ubuntu:/tmp/elastalert# python -m elastalert.elastalert --start NOW --verbose --rule Port_scan_detection.yaml
INFO:elastalert:Starting up
INFO:elastalert:Queried rule Port Scan Detection from 2017-06-22 22:43 IST to 2017-06-22 22:43 IST: 0 / 0 hits
INFO:elastalert:Ran Port Scan Detection from 2017-06-22 22:43 IST to 2017-06-22 22:43 IST: 0 query hits (0 already seen), 0 matches, 0 alerts sent
INFO:elastalert:Sleeping for 4.977606 seconds
INFO:elastalert:Queried rule Port Scan Detection from 2017-06-22 22:43 IST to 2017-06-22 22:43 IST: 0 / 0 hits
INFO:elastalert:Ran Port Scan Detection from 2017-06-22 22:43 IST to 2017-06-22 22:43 IST: 0 query hits (0 already seen), 0 matches, 0 alerts sent
INFO:elastalert:Sleeping for 4.984636 seconds
INFO:elastalert:Queried rule Port Scan Detection from 2017-06-22 22:43 IST to 2017-06-22 22:43 IST: 112 / 112 hits
INFO:elastalert:Ran Port Scan Detection from 2017-06-22 22:43 IST to 2017-06-22 22:43 IST: 112 query hits (0 already seen), 0 matches, 0 alerts sent
INFO:elastalert:Sleeping for 4.82014 seconds
INFO:elastalert:Queried rule Port Scan Detection from 2017-06-22 22:43 IST to 2017-06-22 22:43 IST: 112 / 112 hits
INFO:elastalert:Ran Port Scan Detection from 2017-06-22 22:43 IST to 2017-06-22 22:43 IST: 112 query hits (112 already seen), 0 matches, 0 alerts sent
INFO:elastalert:Sleeping for 4.610031 seconds
*******************************************************************************************************My elastAlert Rule
es_host: 192.168.96.141
es_port: 9200
# ElasticSearch Index Name
index: graylog_0
timestamp_field: timestamp
timestamp_type: custom
timestamp_format: '%Y-%m-%d %H:%M:%S.%f'
timestamp_format_expr: 'ts[:23] + ts[26:]'
doc_type: message
writeback_index: elastalert_status
type: cardinality
query_key: [srx-source-address, srx-destination-address]
cardinality_field: srx-destination-port
max_cardinality: 50
timeframe:
seconds: 30
old_query_limit:
seconds: 1
filter:
- term:
srx-rt-flow: "RT_FLOW_SESSION_DENY"
aggregation:
minutes: 1
alert:
- "email"
My Elastisearch log looks like below
{ "_index": "graylog_0", "_type": "message", "_id": "df53ea91-5748-11e7-a486-000c298681bd", "_version": 1, "_score": 1, "_source": { "gl2_remote_ip": "192.168.96.141", "srx-destination-address": "23.23.23.23", "gl2_remote_port": 53774, "streams": [ "000000000000000000000001" ], "source": "172.16.78.2", "message": "Apr 3 20:34:18 172.16.78.2 1 2017-04-03T20:34:17.842Z RT_FLOW - RT_FLOW_SESSION_DENY source-address="172.16.78.21" source-port="56453" destination-address="23.23.23.23" destination-port="17076" service-name="None" protocol-id="17" icmp-type="0" policy-name="default-deny" source-zone-name="corporate" destination-zone-name="highwinds-inet" application="UNKNOWN" nested-application="UNKNOWN" username="N/A" roles="N/A" packet-incoming-interface="xe-4/0/1.10" encrypted="UNKNOWN" reason="policy deny"]", "gl2_source_input": "59317ae66e332b0c4df4f831", "srx-ip-address": "172.16.78.2", "src-icmp-type": "0", "service-name": "None", "srx-destination-port": "17076", "srx-source-port": "56453", "srx-source-zone-name": "corporate", "srx-policy-name": "default-deny", "srx-rt-flow": "RT_FLOW_SESSION_DENY", "gl2_source_node": "6e3e4964-4439-4cbb-8e49-0baf1a006532", "srx-destination-zone-name": "highwinds-inet", "srx-source-address": "172.16.78.21", "srx-protocol-id": "17", "timestamp": "2017-06-22 12:46:55.416" } }
suchandbabu
commented
6 years ago Hi @sathishdsgithub Thanks for your support.
I am looking for this same requirement mentioned in your previous comment. Please help me if you have the solution for this.
My requirement is given below: Rule has to be get alerted if same srcip, same dstip and differant dstports comes in multiple events in last 15 minutes.
Hi,
I try to use the cardinality rule and more especially the max_cardinality.
The rule seems working correctly hit (query) but impossible to "match" the rule.
The rule:
Example:
I'm sure of the cardinality_field, the value exist. Some constraints exist with max_cardinality ? because if i switch to the min_cardinality to test, it seems working correctly.
Thanks
Benoît