Logical error: `Ephemeral node still exists after 90s` (ALTER DELETE)

[KochetovNicolai]

https://s3.amazonaws.com/clickhouse-test-reports/0/98024a723fe9f357ad9deb820ceeb4392cc22d00/stress_test__debug_.html

2024.08.14 02:02:38.284934 [ 4725 ] {656fa9df-168f-40aa-b1a0-fdd5e71c14e2} <Debug> DDLWorker(test_35): Executing query: ALTER TABLE test_35.movement DELETE WHERE (date >= toDateTime('2020-01-22T16:00:00', 'Asia/Istanbul')) AND (date < toDateTime('2020-01-22T17:00:00', 'Asia/Istanbul')) SETTINGS mutations_sync = 2

...

2024.08.14 02:23:32.251595 [ 4254 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Trace> Planner: Query to stage Complete
2024.08.14 02:23:32.253336 [ 4254 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Trace> Planner: Query from stage FetchColumns to stage Complete
2024.08.14 02:23:32.263491 [ 4254 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Debug> test_35.movement (ccb56b7d-fed8-48f1-95ba-1065fa27c929) (SelectExecutor): Key condition: (column 0 in [1579698000, +Inf)), (column 0 in (-Inf, 1579701600]), an
d
2024.08.14 02:23:32.263763 [ 4254 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Trace> test_35.movement (ccb56b7d-fed8-48f1-95ba-1065fa27c929) (SelectExecutor): Filtering marks by primary and secondary keys
2024.08.14 02:23:32.268773 [ 4254 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Trace> test_35.movement (ccb56b7d-fed8-48f1-95ba-1065fa27c929) (SelectExecutor): Running binary search on index range for part all_1_1_1 (124 marks)
2024.08.14 02:23:32.270859 [ 4254 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Trace> test_35.movement (ccb56b7d-fed8-48f1-95ba-1065fa27c929) (SelectExecutor): Found (LEFT) boundary mark: 85
2024.08.14 02:23:32.275865 [ 4254 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Trace> test_35.movement (ccb56b7d-fed8-48f1-95ba-1065fa27c929) (SelectExecutor): Found (RIGHT) boundary mark: 96
2024.08.14 02:23:32.284132 [ 4254 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Trace> test_35.movement (ccb56b7d-fed8-48f1-95ba-1065fa27c929) (SelectExecutor): Found continuous range in 12 steps
2024.08.14 02:23:32.285235 [ 4254 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Debug> test_35.movement (ccb56b7d-fed8-48f1-95ba-1065fa27c929) (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 11/123 marks by primary 
key, 11 marks to read from 1 ranges
2024.08.14 02:23:32.286083 [ 4254 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Trace> test_35.movement (ccb56b7d-fed8-48f1-95ba-1065fa27c929) (SelectExecutor): Spreading mark ranges among streams (default reading)
2024.08.14 02:23:32.287381 [ 4254 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Test> MergeTreeReadPoolBase: Will use min_marks_per_task=24
2024.08.14 02:23:32.287488 [ 55026 ] {} <Fatal> : Logical error: 'Ephemeral node /test/test_33/rdb/replicas/s9|r9/active still exists after 90s, probably it's owned by someone else. Either session_timeout_ms in client's config is different from server's config or it's a bug. Node data: '''.
2024.08.14 02:23:32.288395 [ 4254 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Trace> test_35.movement (ccb56b7d-fed8-48f1-95ba-1065fa27c929) (SelectExecutor): Reading 1 ranges in order from part all_1_1_1, approx. 90112 rows starting from 69632
0
2024.08.14 02:23:32.309619 [ 4846 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Test> MergeTreeRangeReader: First reader returned: num_rows: 65409, columns: 1, total_rows_per_granule: 65409, no filter, column[0]:  UInt32(size = 65409), requested 
columns: date
2024.08.14 02:23:32.317310 [ 4846 ] {ccb56b7d-fed8-48f1-95ba-1065fa27c929::all_1_1_1_2} <Test> MergeTreeRangeReader: read() returned num_rows: 65409, columns: 1, total_rows_per_granule: 65409, no filter, column[0]:  UInt32(size = 65409), sample block date
2024.08.14 02:23:32.320856 [ 55026 ] {} <Fatal> : Stack trace (when copying this message, always include the lines below):

0. /build/contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x00000000163462d2
1. /build/src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c5f89b9
2. /build/contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000006fd616c
3. /build/contrib/llvm-project/libcxx/include/vector:438: DB::Exception::Exception<String const&, int, String&>(int, FormatStringHelperImpl<std::type_identity<String const&>::type, std::type_identity<int>::type, std::type_identity<String&>::type>, String 
const&, int&&, String&) @ 0x000000001333c62b
4. /build/src/Common/ZooKeeper/ZooKeeper.cpp:0: zkutil::ZooKeeper::deleteEphemeralNodeIfContentMatches(String const&, String const&) @ 0x0000000013331463
5. /build/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:801: DB::DatabaseReplicatedDDLWorker::initializeReplication() @ 0x0000000010d43bed
6. /build/contrib/llvm-project/libcxx/include/atomic:945: DB::DatabaseReplicatedDDLWorker::initializeMainThread() @ 0x0000000010d42fe8
7. /build/src/Interpreters/DDLWorker.cpp:1156: DB::DDLWorker::runMainThread() @ 0x0000000011089111
8. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:0: ? @ 0x00000000110a04a2
9. /build/base/base/../base/wide_integer_impl.h:817: ThreadPoolImpl<std::thread>::worker(std::__list_iterator<std::thread, void*>) @ 0x000000000c69d3f6
10. /build/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:302: void* std::__thread_proxy[abi:v15007]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c6a1825
11. ? @ 0x00007ff3433cdac3
12. ? @ 0x00007ff34345f850

[kssenii]

https://s3.amazonaws.com/clickhouse-test-reports/71176/623b2f11d30af6ff0d00caa56bffbd4590bc4fff/stress_test__debug_/fatal_messages.txt