Expect all log messages recv'd to be indexed into proper index.
Error messages in logstash-plain.log have the following error:
'''
[2018-05-08T16:49:24,944][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"traffic-2018.05.08", :_type=>"doc", :_routing=>nil}, #], :response=>{"index"=>{"_index"=>"traffic-2018.05.08", "_type"=>"doc", "_id"=>"Cz-oQGMBbaX6NU0jWWDP", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [DestinationIPGeo.postal_code.keyword]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Value [94043] is out of range for a short"}}}}}
'''
This is because the DestinationIPGeo.postal_code.keyword is being cast as a short in the logstash conf.d file but it needs to be either an int or a long. More research and testing needs to be done.
Expected behavior and actual behavior.
Expect all log messages recv'd to be indexed into proper index. Error messages in logstash-plain.log have the following error: ''' [2018-05-08T16:49:24,944][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"traffic-2018.05.08", :_type=>"doc", :_routing=>nil}, #], :response=>{"index"=>{"_index"=>"traffic-2018.05.08", "_type"=>"doc", "_id"=>"Cz-oQGMBbaX6NU0jWWDP", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [DestinationIPGeo.postal_code.keyword]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Value [94043] is out of range for a short"}}}}}
'''
This is because the DestinationIPGeo.postal_code.keyword is being cast as a short in the logstash conf.d file but it needs to be either an int or a long. More research and testing needs to be done.