Open mfe5003 opened 7 years ago
Thanks for logging this. You should be able to workaround this issue in the sink connector for the released versions by using an SMT in the sink connector that "corrects" the name of the topic in each sink record before Connect passes them to the connector. You could use the RegexRouter
SMT that comes with Connect (see the Kafka documentation). That uses a matching regex and replaces the first occurrence with a specified value. If for some reason that doesn't work or is not sufficient, you could write a custom SMT.
a friendlier example: https://docs.confluent.io/current/connect/transforms/regexrouter.html
Summary: If there is a period "." in the topic name (like when a namespace is used) the MySQL JDBC connector fails, while the SQLite connector is fine.
I have a namespace
data.*
with a single topicdata.test_stream_0
at the moment. If I use the sqlite3 connector with the following.properties
file contents:The connector creates the
test.db
file (when I start a python AvroProducer withtopic=data.test_stream_0
) and thedata.test_stream_0
table and I can select the entries in the table as I expect.I then switch to mysql (v5.7.19) with jdbc connector (v5.1.38) and
.properties
file contents:After starting a producer, the connector creates the table correctly (confirmed with the mysql client), but experiences an error:
I have traced the discrepancy down to the following two log lines, where the SQLite connector is correctly reading in the table columns after it creates the table, and the MySQL is not.
SQLite:
MySQL:
The MySQL connector works correctly if I use a topic that does not have a period in it, so it seems to me to be an issue with escaping the table name, but the issue does not occur if I use another deliminator that also requires escaping in MySQL such as '-'.
Is there a way to replace the period in the topic with an underscore or some other symbol, that would be an appropriate work around?