TransportError(400, u'search_phase_execution_exception')

[0x9090]

We have alerts which look for unexpected machine logons from certain users. This one rule we have keeps throwing a "search_phase_execution_exception" and I can't seem to figure out why.

The error,

INFO:elastalert:Ran ????? Workstation Suspicious Login from 2017-02-07 17:12 PST to 2017-02-07 17:17 PST: 0 query hits, 0 matches, 0 alerts sent
WARNING:elasticsearch:GET /logstash-2017.02.08/_search?_source_include=%40timestamp%2C%2A&ignore_unavailable=true&size=10000 [status:400 request:0.011s]
ERROR:root:Error running query: TransportError(400, u'search_phase_execution_exception')

The alert,

es_host: ???
es_port: 9200

name: ??? Workstation Suspicious Login

type: frequency

index: logstash-%Y.%m.%d
use_strftime_index: true

num_events: 1

timeframe:
  minutes: 5

use_count_query: true
doc_type: "eventlog"

filter:
- query:
    query_string:
      query: >-
        eventlog_id:(4624)
        AND (LogonType:(2)
          OR LogonType:(3)
          OR LogonType:(10)
          OR LogonType:(11))
        AND (IpAddress.keyword:("???")
          OR WorkstationName.keyword:("???"))
        AND NOT AccountName:("???")
        AND NOT TargetUserName:("???")
        AND NOT AccountName.keyword:("???")

alert:
- "email"

alert_subject: "ElastAlert - Suspicious Login To ??? Computer"

smtp_host: "???"
from_addr: "???"
email:
- "???@domain.tld"
bcc:
- "???@domain.tld"
- "???@domain.tld"

jira_server: "https://???/"
jira_project: "???"
jira_issuetype: "Task"
jira_account_file: "???.yaml"
jira_label: "machine"
jira_bump_tickets: true
jira_bump_not_in_statuses:
- "Resolved"
- "Closed"
- "Test\ Ignore"
- "False\ Positive"

Strange thing - this alert appears to be firing as expected, despite this error. But I'd like to understand why we're seeing this

[wirecutter313]

I'm having a similar issue with a less complex query.

Running ES 5

[Qmando]

You can add --es_debug_trace queries.log to see the exact query being made. It will output them as curl commands. Run it manually and you can see the full error.

[0x9090]

I copied the CURL command, and ran it on the ElasticSearch node, and it gave me

curl: (52) Empty reply from server

Putting the query string into Kibana seems to work fine.