Open amruss opened 1 year ago
cc @cockroachdb/cdc
At the moment, all we allow is an initial scan on the table. We also run the same changefeed statement every time.
There is a use for scheduled/incremental exports: running a changefeed during off-hours to not disrupt SQL traffic. Some code needs to be added to actually make the scheduled run incremental (ie. maybe use CURSOR + end_time or transformatinos with protected timestamp management).
As of 23.1, we will offer incremental exports with CDC Transformations + scheduled changefeeds. Unfortunately, we will not protect the data between exports, so you will miss messages. We should have an option to protect this data.
See scheduled backups for prior art:
Jira issue: CRDB-23892
Epic CRDB-24867