bug: altering table column failed when table is built upon a shared cdc source

yezizp2012 commented 5 months ago

Describe the bug

Altering table column failed when table is built upon a shared cdc source.

dev=> CREATE SOURCE test_db WITH (connector = 'mysql-cdc', hostname = 'localhost', port = '3306', username = 'root', password = '', database.name = 'test', server.id = '5888', debezium.snapshot.mode = 'schema_only_recovery', debezium.snapshot.locking.mode = 'none');
CREATE_SOURCE
dev=> CREATE TABLE cdc_object (oid INT, obj_type VARCHAR, PRIMARY KEY (oid)) WITH (snapshot = true) FROM test_db TABLE 'test.object';
CREATE_TABLE
dev=> alter table cdc_object add column owner_id int;
ERROR:  Failed to run the query

Caused by these errors (recent errors listed first):
  1: gRPC request to meta service failed: Internal error
  2: Different number of internal tables. New: 0, Old: 1

Error message/log

No response

To Reproduce

No response

Expected behavior

No response

How did you deploy RisingWave?

No response

The version of RisingWave

No response

Additional context

No response

yezizp2012 commented 5 months ago

The root cause is that the cdc table info in CreateTable statement is unhandled, which result in wrong streaming graph for new table. Then the internal tables will be mismatched. Cc @StrikeW @BugenZhao .

We should pass the correct streaming graph to meta.
When building the fragment graph, it is necessary to add the actor information of the upstream source. Need to add a new function in CompleteStreamFragmentGraph like with_upstreams_and_downstreams.
Add necessary altering tests.

btw, include_column_options is also not handled.

yezizp2012 commented 5 months ago

Any updates? 👀

StrikeW commented 5 months ago

Not yet. I am busy with adapting postgres data types last week. Will take a look soon.

StrikeW commented 5 months ago

Any updates? 👀

If it is required by a POC user, I suggest you to note it down in the Notion. @yezizp2012

yezizp2012 commented 5 months ago

Any updates? 👀

If it is required by a POC user, I suggest you to note it down in the Notion. @yezizp2012

Nope, it was just reported from community. You can take your time.

KeXiangWang commented 4 months ago

I'll take this but do it later. Still busy with sql server cdc recently.

risingwavelabs / risingwave