alibaba / canal

阿里巴巴 MySQL binlog 增量订阅&消费组件
Apache License 2.0
28.51k stars 7.62k forks source link

adapter 并发执行sql,发生死锁问题 #2818

Open deeXW opened 4 years ago

deeXW commented 4 years ago

client-adapter --rdb 此模块下,在执行sql前,会根据设定的thread数,将targetPk进行hash分组,从而实现多个session并发执行sql。 futures.add(executorThreads[i].submit(() -> { try { dmlsPartition[j] .forEach(syncItem -> sync(batchExecutors[j], syncItem.config, syncItem.singleDml)); dmlsPartition[j].clear(); batchExecutors[j].commit(); return true; } catch (Throwable e) { batchExecutors[j].rollback(); throw new RuntimeException(e); } })); 在测试过程中,Mysql隔离级别为RC,执行以下sql语句3三遍,同一张表,binlog获取所有记录,并发执行,会偶尔出现死锁问题。 DELETE FROM xxx WHERE a = 1 AND b = 10; (a,b为复合索引) DELETE FROM xxx WHERE a = 2 AND b = 10; DELETE FROM xxx WHERE a = 3 AND b = 10; 。。。。 INSERT INTO xxx(a,b,c,...) VALUES (1,10,111) INSERT INTO xxx(a,b,c,...) VALUES (2,10,222) INSERT INTO xxx(a,b,c,...) VALUES (3,10,333) 。。。。

yilongchuan commented 3 years ago

持续关注此问题

fuzhougongxing commented 3 years ago

client-adapter --rdb 此模块下,在执行sql前,会根据设定的thread数,将targetPk进行hash分组,从而实现多个session并发执行sql。 futures.add(executorThreads[i].submit(() -> { try { dmlsPartition[j] .forEach(syncItem -> sync(batchExecutors[j], syncItem.config, syncItem.singleDml)); dmlsPartition[j].clear(); batchExecutors[j].commit(); return true; } catch (Throwable e) { batchExecutors[j].rollback(); throw new RuntimeException(e); } })); 在测试过程中,Mysql隔离级别为RC,执行以下sql语句3三遍,同一张表,binlog获取所有记录,并发执行,会偶尔出现死锁问题。 DELETE FROM xxx WHERE a = 1 AND b = 10; (a,b为复合索引) DELETE FROM xxx WHERE a = 2 AND b = 10; DELETE FROM xxx WHERE a = 3 AND b = 10; 。。。。 INSERT INTO xxx(a,b,c,...) VALUES (1,10,111) INSERT INTO xxx(a,b,c,...) VALUES (2,10,222) INSERT INTO xxx(a,b,c,...) VALUES (3,10,333) 。。。。

临时解决方案,调整hash规则,根据数据表进行hash