scylladb / scylla-cdc-source-connector

A Kafka source connector capturing Scylla CDC changes
Apache License 2.0
46 stars 18 forks source link

feature request: allow to define initial ChangeAgeLimit #17

Open hartmut-co-uk opened 2 years ago

hartmut-co-uk commented 2 years ago

As a connect user/admin I'd like to be able to configure scylla.change.age.limit.

(quoting scylla-cdc-go)

When the library starts for the first time it has to start consuming changes from some point in time. This parameter defines how far in the past it needs to look. If the value of the parameter is set to an hour, then the library will only read historical changes that are no older than an hour.

Consuming from a 'connector::table::vnodeId::streamId' should start from ~ max(persistedState, now()-tableCdcTtl, now()-changeAgeLimit)

References