search

ibm-messaging / kafka-connect-mq-sink

This repository contains a Kafka Connect sink connector for copying data from Apache Kafka into IBM MQ.
Apache License 2.0
35 stars 43 forks source link

Sending messages to IBM MQ with a different encode than UTF-8 #42

Open brenofr opened 3 years ago

brenofr commented 3 years ago

Hello!

I have 2 points here:

  1. I'd like to know if there a way to send a message to a IBM MQ queue using a different encode (like "IBM500"). In my need, I noticed that there's data loss caused by the differences between a content that is received as EBCDIC (IBM500) and the default value used by IBM MQ Sink connector is UTF-8.

  2. I'd like to know if is it possible to input a custom source code into the connector to force the ccsid.

PS.: I can't change the CCSID of que Queue Manager.

jhughes24816 commented 1 year ago

Hi @brenofr sorry it has taken so long to respond to this.

  1. Some details of message format options can be found in this section of the README which give some details about message formatting however is no reference to supporting alternative encodings.
  2. At the moment, this is not possible as the encoding is fixed to UTF-8. It might be possible to expose this as part of the connector config.

I will raise it with the development team.

theodore-evans commented 11 months ago

Hi @brenofr, @jhughes24816 fyi,

I have just opened a PR on the IBM MQ source connector repo to add support for configuring the connector with a non-default CCSID: https://github.com/ibm-messaging/kafka-connect-mq-source/pull/125

Provided there are no issues with this going through, the same approach could be implemented for the sink connector.