The confluent REST proxy required values to be encoded when submitted. The kafka-node library does this for us. This means that messages published using a library that dealt with the confluent proxy were being double encoded, which caused issues for messages >= 50% of kafka's message.max.bytes setting.
This PR resolves this by testing to see whether the message is base64 encoded before publishing it and decoding it. It also supports consumers specifying whether they want it encoded on the way out (true by default) so that consumers using the same library can continue to function.
The confluent REST proxy required values to be encoded when submitted. The kafka-node library does this for us. This means that messages published using a library that dealt with the confluent proxy were being double encoded, which caused issues for messages >= 50% of kafka's message.max.bytes setting.
This PR resolves this by testing to see whether the message is base64 encoded before publishing it and decoding it. It also supports consumers specifying whether they want it encoded on the way out (true by default) so that consumers using the same library can continue to function.