dm: syncer failed to sync remaining dml after shard-ddl is resolved

D3Hunter commented 1 year ago

What did you do?

prepare 2 source: 3306 and 3307
execute create database test; use test; create table t(a decimal(20,6)) charset utf8mb4 collate utf8mb4_bin; on both source
start dm task, wait it enter sync phase
on 3306, run: alter table test.t modify column a decimal(20,2); insert into test.t values(1); insert into test.t values(1);
on 3307, run: alter table test.t modify column a decimal(20,2);
query-status, no error. show-ddl-locks shows ddl is synced
select * from test.t; on downstream, empty result while we expect 2 rows
on 3306, run insert into test.t values(1);
select * from test.t; on downstream, shows 3 rows

task config

name: test
task-mode: all
is-sharding: true
shard-mode: optimistic
ignore-checking-items: ["all"]

target-database:
  host: "127.0.0.1"
  port: 4000
  user: "root"
  password: ""

mysql-instances:
  - source-id: "mysql-3306"
    block-allow-list:  "ba-rule1"
    route-rules: ["global"]
    mydumper-config-name: "dump1"
    loader-config-name: "load1"
  - source-id: "mysql-3307"
    block-allow-list:  "ba-rule1"
    route-rules: ["global"]
    mydumper-config-name: "dump2"
    loader-config-name: "load2"

block-allow-list:
  ba-rule1:
    do-dbs: ["test"]
mydumpers:
  dump1:
    mydumper-path: "dumped_data-dump1"
  dump2:
    mydumper-path: "dumped_data-dump2"
loaders:
  load1:
    dir: "loader_load1"
  load2:
    dir: "loader_load2"

routes:
  global:
    schema-pattern: test
    table-pattern: "t*"
    target-schema: test
    target-table: "t"

What did you expect to see?

No response

What did you see instead?

No response

Versions of the cluster

DM version (run dmctl -V or dm-worker -V or dm-master -V):

6.5.0

Upstream MySQL/MariaDB server version:

one 5.7, one 8.0

Downstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

6.1

How did you deploy DM: tiup or manually?

(leave TiUP or manually here)

Other interesting information (system version, hardware config, etc):

>
>

current status of DM cluster (execute `query-status <task-name>` in dmctl)

(paste current status of DM cluster here)

fubinzh commented 1 year ago

/severity major

fubinzh commented 1 year ago

This could be a problem, If user wants to wait for the 2 DML synced to downstream before other DMLs.

D3Hunter commented 1 year ago

very rare case in real environment, change to minor

pingcap / tiflow