search

logstash-plugins / logstash-input-sqs

Apache License 2.0
16 stars 40 forks source link

Plugin drops data when multiple objects are returned from codec.decode (json_lines) #26

Closed packplusplus closed 2 years ago

packplusplus commented 8 years ago

Bug report + code fix: https://discuss.elastic.co/t/sqs-input-plugin-only-pulls-one-json-object-from-sqs-payload/50352/3

TLDR; When using a codec to created multiple events from a single sqs payload, only a single event would be ingested.

The way the code was structured would return on the first object codec.decode returned. By flattening some of the code, the json_lines codec worked again. Do you want a PR with the flattened fix, or do you have a different way you'd like to handle this?

ph commented 8 years ago

@packplusplus Yup this look like a bug, would you mind creating a PR for it?

ph commented 8 years ago

I am +1 on the proposed fix, but I will have to test it with sqs to make sure it work. Lets make sure we remove the unused methods too.

packplusplus commented 8 years ago

Not sure how to add/update/run the rspec testing. I think it would go here, but I'm not really setup to run them (and don't particularly know how to code them). I think you'd pass in a message with 3 json objects in them, and then count how many events it created.

edit: annnnnnnd....it breaks the rspec. How would you want to fix this? We could move 135/136 out to their own function and call that?

ph commented 8 years ago

@packplusplus I can take your PR from there and fix the test I will keep attribution of your commit, does that help?

packplusplus commented 8 years ago

However you wanna role PH! Not too caught up in attribution, credit, just trying to smash bugs. I wouldn't mind seeing a doc where they explain how to test ls plugins so I could be a better contributor next time.

packplusplus commented 2 years ago

Closing, I lost context of if this is still an issue and don't want to clutter up the issues history.