bpaquet / node-logstash

Simple logstash implmentation in nodejs : file log collection, sent with zeromq
Other
517 stars 141 forks source link

Elasticsearch bulk operation lose data #101

Closed xylvxy closed 9 years ago

xylvxy commented 9 years ago

According to Elasticsearch Api doc, bulk api post data must like this:

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n

ES Bulk Api

But current master branch, lack of one '\n'. As a result, the last line data is ignore by Elasticsearch. node-logstash/lib/lib/elastic_search_helper.js

exports.buildBulkPayload = function(data) {
  return data.map(function(x) {return '{"index": {}}\n' + JSON.stringify(x);}).join('\n');
};

So, anyone could fix this problem. THX