search

logstash-plugins / logstash-output-sqs

Apache License 2.0
4 stars 22 forks source link

Add max size handling of mesages #4

Closed gbutt closed 8 years ago

gbutt commented 9 years ago

Adds a setting (message_max_size) to drop all messages that exceed this size. This setting defaults to 256kb because SQS will reject any message over 256kb in size.

Adds a setting (stop_on_fail) that prevents stud/buffer from forcing infinite retries when SQS rejects a message. Defaults to false, but it should probably default to true in order to avoid racking up infinite SQS charges. Another strategy would be to specify a max number of retries and default that to a reasonable number.

erikthorselius commented 9 years ago

:+1:

erikthorselius commented 9 years ago

@ph maybe you could have a look on this? I have some nasty exception happening in a subsystem and it would be nice that logstash don't die because of this :-).

erikthorselius commented 9 years ago

What happened on this?

erikthorselius commented 9 years ago

Still no news?

suyograo commented 9 years ago

@gbutt can you please sign the CLA? https://github.com/elasticsearch/logstash/blob/master/CONTRIBUTING.md#contribution-steps

elasticsearch-release commented 9 years ago

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then say 'jenkins, test it'.

lloydpick commented 8 years ago

We've just encountered this problem today, is there any movement on this?

jrgns commented 8 years ago

@lloydpick I'll try and look at this this week, and clean it up so that we can get it into the plugin.

oillio commented 8 years ago

Any update on this?

I ran into this as well. I implemented my own fix for this before I came across this PR. If the missing CLA is what is holding up merging this in, I can submit a separate PR with my fix.

joshuaspence commented 8 years ago

It would be great if this could be merged.

martinssipenko commented 8 years ago

Any updates on this?

gbutt commented 8 years ago

Strange I thought I signed the CLA a while ago. Just so you know my company got acquired by CA and they made us sign over all IP rights. If I were you I wouldn't touch this pull request with a 10-foot pole.

karmi commented 8 years ago

Hi @gbutt, we have found your signature in our records, but it seems like you have signed with a different e-mail than the one used in yout Git commit. Can you please add both of these e-mails into your Github profile (they can be hidden), so we can match your e-mails to your Github profile?

gbutt commented 8 years ago

@karmi, unfortunately I cannot associate my gmail address because it is associated with another github account that I cannot deactivate for specific reasons. I signed the CLA moments ago with my rallydev email, however I would not recommend accepting a pull request from my github account while I am employed by CA, it's not worth the risk.

martinssipenko commented 8 years ago

There is another PR #16 with same code changes, can that be merged? Or that also can't be accepted due to CLA/CA ?

cc @gbutt