scylladb / kafka-connect-scylladb

Kafka Connect Scylladb Sink
Apache License 2.0
42 stars 22 forks source link

Batching and localdc support #3

Closed patitapaban19 closed 4 years ago

patitapaban19 commented 4 years ago

Added support for the following:

  1. scylladb.max.batch.size
  2. scylladb.loadbalancing.localdc And documentation improvements for CONFIG.md.
tarzanek commented 4 years ago

see https://docs.datastax.com/en/drivers/java/3.7/com/datastax/driver/core/BatchStatement.html#add-com.datastax.driver.core.Statement-

tarzanek commented 4 years ago

for the timestamp issue I think we should open a talk to review the design from where I stand it looks like we should keep timestamp of every single record from kafka(as source of truth) so I guess the user should know when he starts using batching, that his timestamps will get the same or another solution is to add a specific kafka timestamp column