peterprib / node-red-contrib-kafka-manager

Implement
GNU General Public License v3.0
22 stars 12 forks source link

Kafka producer v0.6.2 producing null kafka messages from stringified json #61

Open DsmythMI opened 1 year ago

DsmythMI commented 1 year ago

"Kafka producer" in v0.6.2 producing "null" kafka messages from stringified json payload. Reverting to v0.5.1 resolves the issue
flow kafka_nulls

peterprib commented 1 year ago

I did do some major uplift of payload adding parameters that may be having wrong default behavior on upgrade of code. Around compression and fixed some potential problems.

What is not clear to me from what is sent and stated, the kafka node is at the end point of flow. Debug nodes imply this is occurring before the kafka node thus how could it be causing the issue.

If it is a problem with default values this is easily identified by editing the node and checking the values are as expected and saving if they needed to be modified.

DsmythMI commented 1 year ago

Hi There, The "Kafka producer" node is pretty much configured with the defaults, the node json is attached (see file).

kafka_producer_node_json.txt

The "stringify json" node stringify_function_node.txt

sends the msg.payload containing a string

msg_payload.txt

The "kafka producer" node in v0.5.1 seems to then correctly create a message in the kafka topic with the json string. Whereas at v0.6.2 a "null" message is created in the topic

peterprib commented 1 year ago

The way topic is set

const msgTopic = node.topicSlash2dot && msg.topic ? msg.topic.replace('/', '.') : msg.topic
const sendMsg = {
  topic: msgTopic || node.topic || '',

which implies the message being sent has the property topic and may be the issue. Is this property set? As re-arranged a bit, not sure if I made incoming msg topic take precedence over set topic and this could be the issue. Can remember thinkng about adding a option to always set to node.topic.

Note, produce has a option to stringify a payload which is JSON. Not sure when I put it in.