cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.11k stars 3.81k forks source link

cdc: create changefeed from previous changefeed #69348

Open shermanCRL opened 3 years ago

shermanCRL commented 3 years ago

Is your feature request related to a problem? Please describe.

If a changefeed has failed, and an operator wishes to create a new identically-configured changefeed, it is tedious to retrieve the previous changefeed CREATE command. (Selecting against SHOW JOBS, then selecting the command, etc).

Describe the solution you'd like

Syntax to allow creating a new changefeed, given a job id(s) for the failed changefeed.

Maybe CREATE CHANGEFEED FROM JOB 123456, but I don’t have strong opinions on the syntax particulars.

Jira issue: CRDB-9553

shermanCRL commented 3 years ago

Need to think about security concerns here. Would copying a job provide a path to see credentials created by someone else?

shermanCRL commented 3 years ago

Possible this is better addressed via CREATE CHANGEFEED FROM TEMPLATE?

Syntax idea from @HonoreDB : CREATE CHANGEFEED LIKE, similar to CREATE TABLE LIKE.

miretskiy commented 1 year ago

I'm inclined to close this issue; Ideally, changefeeds should not fail -- and there has been a lot of progress in that space. So, such a feature would have very limited use case in my opinion.

@shermanCRL -- please close this issue, or keep it open if you disagree.

shermanCRL commented 1 year ago

We can probably keep this idea in the backlog -- I think it makes sense from a “fleet management” perspective, i.e. “I need to create a bunch of changefeeds and can you save me some tedium”. But it’s sugar, like copy-paste, and may not have an actual customer advocates, I will ask around.

Agree that it’s become less relevant as a way of recovering from failures, which was an initial motivation.

Actually, maybe SHOW CREATE CHANGEFEED <jobid> is a better answer.