search

bpaquet / node-logstash

Simple logstash implmentation in nodejs : file log collection, sent with zeromq
Other
517 stars 141 forks source link

Unserializers raw => Make me fail? #105

Closed janober closed 9 years ago

janober commented 9 years ago

Just started to play around with node-logstash. But was totally unable to read from rabbitMQ. No matter what I did it was never able to parse the data.

After looking through the code I think I found now the issue. For me it looks like that the unserializer "raw" (which is the one I need) can not work at all. Currently it says in the code:

BaseInput.prototype.unserialize_data_raw = function(data, ok_callback, parse_fail_callback) {
  parse_fail_callback(data);
};

So with other words: "No matter what you give me I fail" which is the exact opposite of what I would have expected.

Should it not call "ok_callback" instead?

bpaquet commented 9 years ago

RabbitMQ is only a transport. You can use raw or json_logstash format through it. If you use raw, you loose metadata (original timestamp, filename origin, host origin ...).

You have to use the same format to RabbitMQ and from RabbitMQ.

The code for raw format is correct : it always fail to parse data, because we want no data parsing, only raw data.

On Thu, Jun 4, 2015 at 2:16 AM, Jan notifications@github.com wrote:

Just started to play around with node-logstash. But was totally unable to read from rabbitMQ. No matter what I did it was never able to parse the data.

After looking through the code I think I found now the issue. For me it looks like that the unserializer "raw" (which is the one I need) can not work at all. Currently it says in the code:

BaseInput.prototype.unserialize_data_raw = function(data, ok_callback, parse_fail_callback) { parse_fail_callback(data); };

So with other words: "No matter what you give me I fail" which is the exact opposite of what I would have expected.

Should it not call "ok_callback" instead?

— Reply to this email directly or view it on GitHub https://github.com/bpaquet/node-logstash/issues/105.

janober commented 9 years ago

Thanks for the explanation of raw! Turned out that the problem why "json_logstash" did not work was that I had no "@timestamp" attribute. So close...