Support copying of JSON data to pubsub topic as JSON instead of stringified map.

[sriramkoopidai]

Problem: The issue is with the sink connector, If schema is not used, JSON data in topic is pushed to pubsub topic as string value of the map. We need a feature so that we can force the output to be JSON and could be interpreted as JSON in pubsub.

Example for JSON object {"name":"bob","id":45}

it is shown as on pubsub topic {name=bob, id=45}

Add a configuration which could be used to force output to be JSON.

Alternativies considered: Tried forcing the output to be string using configuration value.converter=org.apache.kafka.connect.storage.StringConverter

pubsub topic gets the right value but the single message transforms didn't not work, for example if you want to copy json field to header, it does not work.

Example: Sample SMT

"headers.publish" : true, "transforms": "insertAppIdHeader,moveFieldsToHeader", "transforms.moveFieldsToHeader.type": "org.apache.kafka.connect.transforms.HeaderFrom$Value", "transforms.moveFieldsToHeader.fields": "name,id", "transforms.moveFieldsToHeader.headers": "name,id",

[lqiu96]

Do you have a repro that we could follow/ use?

CC: @anguillanneuf who may have more context regarding this issue.

[samarthsingal]

Could you share the sink connector config you are using?

[ThomasBarach]

Hello, I'm hitting this issue as well. Did you manage to do something about it? Regards

[CatherineThompson]

Hi, I've also run into this issue. Is there a workaround?