Open bvt123 opened 9 months ago
test:
create table t4 on cluster '{cluster}' (a UInt32, pos DateTime) engine=ReplicatedMergeTree order by pos;
system stop fetches on cluster '{cluster}' t4;
insert into t4 values (1, now()); -- node 0
insert into t4 values (2, now()); -- node 1
insert into t4 values (3, now()); -- node 0
select *,_part from t4 order by pos;
select * from system.replication_queue;
with splitByChar('_',_part) as block
select checkBlockSequence(groupUniqArray([toUInt32(block[2]),toUInt32(block[3])])) as a from t4;
system start fetches on cluster '{cluster}' t4;
select checkBlockSequence(groupUniqArray([toUInt32(block[2]),toUInt32(block[3])])) as a from t4;
For clustered setups where the event stream (like from Kafka) goes to different replicas, some rows from the source table could be lost if replication lag became greater than the delay configured per destination table.
We already have a workaround for that situation in some processing Templates:
But it looks ugly. While waiting for table replication to be not active is possible in some cases, checking replication_queue on every cluster node looks very strange.
It could be done better.
When Prepare for Transform code (Template) reads data from the source table it can check Clickhouse insert blocks sequence in all parts that are applied to the WHERE filter. Part names contain block numbers and if some part is still not fetched from the remote replica block sequence will be not monotonic and the error or warning should be fired enabling the retry of the Transform attempt.
The source table should not be partitioned as it complicates checking block sequences.