ERROR:root:Error writing alert info to Elasticsearch: RequestError(400, 'mapper_parsing_exception', "failed to parse field [match_body.weblog.body_bytes_sent] of type [long] in document with id 'iiC6m3MBDLealW5s7dvq'")

[drboone]

I did not create the schema attached below.. I did run the create script at install time.

There are two concerns here:

• The value of the indicated field may be a number, e.g. "123", or it may be "-". Something is making an invalid assumption about data type.
• In my installation, different rules will end up searching different ES indexes with different record structures. What's going to happen when that occurs?

It would help if the documentation talked about how these schemas are established. I haven't been able to find that, but maybe I'm blind.

ES says the schema is: elastalert.txt

ES v6.8.9, Elastalert v0.2.4.

[nsano-rururu]

The following issues may be helpful for past issues.

mapper parsing exception when triggering alert from journalbeat index #2444 mapper_parsing_exception when writing alert info to Elasticsearch #2214 no mappping found for..... #1682 Dots in field names passed to top_count_keys #762

[nsano-rururu]

We also recommend asking questions on the Gitter channel https://gitter.im/Yelp/elastalert

[drboone]

I eventually figured out that the setup script makes the index correctly, but that it occasionally gets remade with a different schema somehow. Attached is the diff that sorted out my problem for elasticsearch v6.

failed_to_parse_field.diff.txt

[nsano-rururu]

@drboone

Should make a pull request

[nsano-rururu]

Does Elasticsearch 6 and 7 need to have different config files?

[drboone]

I don't know. Would make sense. It currently doesn't have mappings for 7 in that subtree.

[mrfroggg]

@drboone , it also fixed my issue as some date field (unix_ms type) and caused alerts not being processed! I'll add this patch my running EA with this. Hopefully, this can be merged in the base code!