Not getting any hits/matches on frequency rule type

[jumanSalameh]

Hey, I'm trying to find out if within an hour, a new document with the field "_id" is in an elasticsearch index I specify. I am using a frequency rule type. normally I receive >4000 documents with an _id field daily. but I am not getting any hits or matches unless I specify --day 5 ! Also, the number of hits is constant for any number of days > 5 which really doesn't make sense since there is not a fixed number of documents added to ES daily.. here's my frequency rule config:

es_host: localhost es_port: 9200 name: test rule type: frequency index: myindex num_events: 1 timeframe: hours: 1

filter: - exists: field: "_id"

alert: - "email"

email: - "my email address"

[Bongsakorn]

Did you try to using elastalert-test-rule? I suggest you to check query that elastalert query in elasticsearch. May be timestamp field does not match.

[alexandreabeh]

I may be missing something here @jumanSalameh but according to the filter documentation 'exists' is not a keyword. Running elastalert via command line with the verbose option, it looks like elastalert shows a QueryParsingException yet still returns 0 hits and 0 matches.

The syntax that has worked for me is filter: - query: wildcard: Level: "*" as an example to get anything with the Level field.

However, there might be a parsing problem with starting a field name with an underscore. I tried both '_id' and '_type' and never got any matches, but fields like 'Level' and 'message' work just fine. So if you're trying a count for all documents, maybe use a different field that is common to all your events.

[jumanSalameh]

@alexandreabeh Thanks a lot for your help, I'll try that soon. I just stopped working on the project for the meantime.