cdc: support jdbc:// url scheme for changefeed

dbist commented 2 years ago

Is your feature request related to a problem? Please describe.

Typically my conversations around CDC involve CockroachDB as source and some sort of OLAP Database as target for analytical purposes. Looking at the database landscape, jdbc:// is a widely accepted method of interacting with an analytical database. Furthermore, CockroachDB supports jdbc:// and it potentially can open up ability to use CockroachDB as destination as well. The goal is to reduce friction in building analytical pipelines. Today, if a user wanted to connect to Snowflake, they would have to create a changefeed to object storage and then use Snow pipe to ingest into Snowflake. With jdbc, they'd be able to connect to it directly.

Describe the solution you'd like

CREATE CHANGEFEED FOR TABLE office_dogs, employees INTO 'jdbc:postgresql//<host>:<port>';

Describe alternatives you've considered

alternative is to create database specific connections, arguably better supportability of the target database but it limits usability to a single sink vs. using a widely accepted driver to connect to any jdbc compatible database.

This concept is similar to https://debezium.io/blog/2017/09/25/streaming-to-another-database/ but it would have an added benefit of decoupling Kafka.

Jira issue: CRDB-15562

Epic CRDB-20421

blathers-crl[bot] commented 2 years ago

cc @cockroachdb/cdc

miretskiy commented 1 year ago

@amruss mind taking a look at this issue? I don't know if we will be spending much time (any?) around additional sink support.

cockroachdb / cockroach

cdc: support jdbc:// url scheme for changefeed #80661