scylladb / kafka-connect-scylladb

Kafka Connect Scylladb Sink
Apache License 2.0
46 stars 22 forks source link

Cannot create table with "." in name #13

Closed matus-cuper closed 2 years ago

matus-cuper commented 4 years ago

Since Kafka allows dot character in topic names (for example my.test.topic) we use them as word separator. Unfortunately, connector attached to the topic with this name will try to create Scylla table dottest.my.test.topic and throw following exception:

...
[2020-05-19 11:27:14,248] INFO create() - Adding table dottest.my.test.topic

    CREATE TABLE dottest.my.test.topic(
        id int,
        firstName varchar,
        lastName varchar,
        PRIMARY KEY(id))
    WITH compression = {'sstable_compression' : ''} (io.connect.scylladb.ScyllaDbSchemaBuilder:341)
[2020-05-19 11:27:14,269] ERROR WorkerSinkTask{id=scylladb-sink-connector-dot-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. Error: line 2:24  : syntax error...
 (org.apache.kafka.connect.runtime.WorkerSinkTask:566)
...

Environment: mvn clean install of coomit aa89618ccbab7aa2d14da67c7b86a305dd1e914d local confluent v5.5.0 local scyllaDB cluster connector definition

{
  "name" : "scylladb-sink-connector-dot",
  "config" : {
    "connector.class" : "io.connect.scylladb.ScyllaDbSinkConnector",
    "tasks.max" : "1",
    "topics" : "my.test.topic",
    "scylladb.contact.points" : "127.0.0.1",
    "scylladb.keyspace" : "dottest",
    "scylladb.keyspace.create.enabled": "true"
  }
}
forsberg commented 4 years ago

Same problem with "-", which is OK in Kafka topics, but not OK in Scylla.