cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.87k stars 3.77k forks source link

CREATE CHANGEFEED attempts to create a "{_family_}" topic in downstream sink #128973

Open smcvey opened 4 weeks ago

smcvey commented 4 weeks ago

Describe the problem

Creating a changefeed on a multi-family table with the split_column_families attempts to create a topic called "{family}" on the broker, despite this not being the name of any column family defined in the table.

To Reproduce

CREATE TABLE t (
    id UUID PRIMARY KEY,
    s STRING,
    FAMILY "a" (id),
    FAMILY "b" (s)); -- two column families, called a and b
CREATE CHANGEFEED FOR TABLE t INTO '<kafka sink address>' WITH OPTIONS (split_column_families);

Without populating the table with any entries, if I go and look at the created topics, I see one that shouldn't be there:

NAME                    PARTITIONS  REPLICAS
t._u007b_family_u007d_  1           1

This topic does not show up on 23.2.x, but it does show up in 24.1.x and 24.2.x. However, all versions (including 23.2.x) show this nonsense notice message:

NOTICE: changefeed will emit to topic t._u007b_family_u007d_

With a broker that doesn't automatically create topics, the changefeed cannot start up and produces the following log message:

Changefeed job <job id> encountered transient error: ‹kafka server: Request was for a topic or partition that does not exist on this broker› (attempt 1)

Expected behavior

It should not attempt to create a {_family_} topic, especially as this topic doesn't receive any messages. This breaks changefeeds that send to a broker that has disabled automatic topic creation as they cannot start up. Also the above NOTICE message should not be produced.

Environment:

Jira issue: CRDB-41314

blathers-crl[bot] commented 4 weeks ago

Hi @smcvey, please add branch-* labels to identify which branch(es) this C-bug affects.

:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

blathers-crl[bot] commented 3 weeks ago

cc @cockroachdb/cdc