Kafka Output support manipulating kafka message headers

[LaithSiriani]

• Add the ability to set headers in the output kafka message
• Add unittest and integration test case to verify

Thanks for contributing to Logstash! If you haven't already signed our CLA, here's a handy link: https://www.elastic.co/contributor-agreement/

[LaithSiriani]

Due to previous failure in the Travis CI build, I modified the commit and uploaded a fix for some flakiness in the "when KafkaProducer#send() raises a non-retriable exception" test case

[LaithSiriani]

@roaksoax Hi, would it be possible to review this pull request? What would be needed to include this feature?

[LaithSiriani]

@mashhurs Sorry for the possibly unnecessary involvement, I saw you had reviewed the last pull request. What is the way of working to get a code review for the pull request?

[LaithSiriani]

Hi @yaauie, @mashhurs , the comments have been addressed.

[mashhurs]

If we don't have an explicit use-case for making the keys variable, then I would suggest avoiding the sprintf on the keys (and changing the docs to reflect that only the values are interpolated).

Thank you so much @LaithSiriani for addressing all comments and moving forward. Just to clarify @yaauie's last comment:

If we don't have an explicit use-case for making the keys variable, then I would suggest avoiding the sprintf on the keys (and changing the docs to reflect that only the values are interpolated).

We would opt out sprintf operation for now if your case doesn't require it.

[LaithSiriani]

Hi @mashhurs , the comment was explicitly about the key, which has been addressed, the value does have a use case to be interpolated by the sprintf

[mashhurs]

New v11.4.0 version released with this change.