fescar-robot
commented
5 years ago Hi @CarolineHuang5954, we detect non-English characters in the issue. This comment is an auto translation from @fescar-robot to help other users to understand this issue. We encourage you to describe your issue in English which is more friendly to other users.
Seata does not support DynamicDataSource
- [ ] I have searched the issues of this repository and believe that this is not a duplicate.
Ⅰ. Issue Description
Currently, distributed data applications use dynamic data sources. Through the operation of dynamic data sources A and B, it is found that the rollback log can be normally inserted into the undo_log table of the corresponding data source, but the distributed transaction cannot be correctly rolled back and submitted. Routing to the corresponding data source to operate on the rollback log and business data
Ⅱ. Describe what happened
Scene description: Application 1:: Transaction Participant, connecting to the database via a dynamic data source, with a default data source Application 2: The transaction initiator, no data source configuration, invokes the interface of application 1, and dynamically operates the data source by passing parameters. Data source A: business table person Data Source B: Business Table user Data source C: default data source, seata-server configures this data source, global_table, branch_table, lock_table in this
Scenario 1: Application 2 performs database operations of A and B by calling the dubbo interface of application 1 twice.
- The new person succeeds and the new user succeeds.
- Data sources A and B are inserted into an undo_log respectively.
- When submitting a global transaction, Application 1 reports an error: the undo_log table of data source C cannot be found (there is no undo_log table in C)
- The undo_log of data source A and B are not deleted.
Scenario 2: Application 2 performs database operations of A and B by calling the dubbo interface of application 1 twice.
- Adding a new user succeeds, adding a new user fails (primary key conflict)
- Data source A inserts an undo log
- When rolling back the global transaction, the application 1 reports an error: the undo_log table of the data source C cannot be found.
- lock_table inserts a piece of data, but the data source of resource_id is C (I think it should be A)
- Global transaction rollback failed, A's undo log is not deleted.
The following is the stack information that can not be found by the undo_log of the C data source. If there is an exception, please attach the exception trace:
2019-08-21 15:57:09.469 INFO 8568 --- [atch_RMROLE_8_8] io.seata.rm.AbstractRMHandler : Branch committing: 172.16.68.197:8091:2020086258 2020086260 jdbc:mysql://172.16.68.118:3306/zhttest null
2019-08-21 15:57:09.469 ERROR 8568 --- [atch_RMROLE_7_8] druid.sql.Statement : {conn-10001, pstmt-20433} execute error. SELECT * FROM undo_log WHERE branch_id = ? AND xid = ? FOR UPDATE
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'zhttest.undo_log' doesn't exist
at sun.reflect.GeneratedConstructorAccessor46.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1054)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4237)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4169)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2617)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2778)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2825)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2156)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2323)
at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_executeQuery(FilterChainImpl.java:3188)
at com.alibaba.druid.filter.FilterEventAdapter.preparedStatement_executeQuery(FilterEventAdapter.java:465)
at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_executeQuery(FilterChainImpl.java:3185)
at com.alibaba.druid.wall.WallFilter.preparedStatement_executeQuery(WallFilter.java:641)
at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_executeQuery(FilterChainImpl.java:3185)
at com.alibaba.druid.filter.FilterEventAdapter.preparedStatement_executeQuery(FilterEventAdapter.java:465)
at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_executeQuery(FilterChainImpl.java:3185)
at com.alibaba.druid.proxy.jdbc.PreparedStatementProxyImpl.executeQuery(PreparedStatementProxyImpl.java:181)
at com.alibaba.druid.pool.DruidPooledPreparedStatement.executeQuery(DruidPooledPreparedStatement.java:228)
at io.seata.rm.datasource.undo.UndoLogManager.undo(UndoLogManager.java:154)
at io.seata.rm.datasource.DataSourceManager.branchRollback(DataSourceManager.java:185)
at io.seata.rm.AbstractRMHandler.doBranchRollback(AbstractRMHandler.java:124)
at io.seata.rm.AbstractRMHandler$2.execute(AbstractRMHandler.java:68)
at io.seata.rm.AbstractRMHandler$2.execute(AbstractRMHandler.java:64)
at io.seata.core.exception.AbstractExceptionHandler.exceptionHandleTemplate(AbstractExceptionHandler.java:117)
at io.seata.rm.AbstractRMHandler.handle(AbstractRMHandler.java:64)
at io.seata.rm.DefaultRMHandler.handle(DefaultRMHandler.java:63)
at io.seata.core.protocol.transaction.BranchRollbackRequest.handle(BranchRollbackRequest.java:35)
at io.seata.rm.AbstractRMHandler.onRequest(AbstractRMHandler.java:149)
at io.seata.core.rpc.netty.RmMessageListener.handleBranchRollback(RmMessageListener.java:81)
at io.seata.core.rpc.netty.RmMessageListener.onMessage(RmMessageListener.java:71)
at io.seata.core.rpc.netty.AbstractRpcRemotingClient.dispatch(AbstractRpcRemotingClient.java:166)
at io.seata.core.rpc.netty.AbstractRpcRemoting$3.run(AbstractRpcRemoting.java:371)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)Ⅲ. Describe what you expected to happen
- The undo log can be inserted into the corresponding data source normally. Then, when the global transaction commits or rolls back, the corresponding data source should be found to operate on the rollback log (directly delete, or delete).
- The current dynamic data source implementation in the application is as follows, hoping to help improve distributed transaction support for dynamic data sources.
All data sources are manually registered by the beanFactory at application startup (query from the data source service, support for dynamic additions and changes, no configuration)
Map<Object, Object> targetDataSources = new HashMap<>();
// Add the primary data source to more data sources
targetDataSources.put("dataSource", defaultDataSource);
// Add more data sources
targetDataSources.putAll(dynamicDataSources);
// Create a DynamicDataSource
GenericBeanDefinition beanDefinition = new GenericBeanDefinition();
beanDefinition.setBeanClass(DynamicDataSource.class);
beanDefinition.setSynthetic(true);
MutablePropertyValues mpv = beanDefinition.getPropertyValues();
mpv.addPropertyValue("defaultTargetDataSource", defaultDataSource);
mpv.addPropertyValue("targetDataSources", targetDataSources);
beanFactory.registerBeanDefinition("dataSource", beanDefinition);
Logger.info("Multiple data source registration success");Get the dataSource through the applicationContext in SeataAutoConfig.java
@Autowired
private ApplicationContext applicationContext;
@Bean
@Primary
public DataSourceProxy dataSourceProxy(DataSource dataSource ){
return new DataSourceProxy(dataSource);
}Dynamically set the data source by adding a slice in the DAO layer and changing the thread variable dataSourceType
public class DynamicDataSource extends AbstractRoutingDataSource {
private static final ThreadLocal<String> contextHolder = new ThreadLocal<>();
private static Map<Object, Object> dynamicDataSources = new HashMap<>();
public static Map<String, DataSource> dynamicDataSourcesPublic = new HashMap<>();
@Override
protected Object determineCurrentLookupKey() {
String dataSourceType = getDataSourceType();
if (dataSourceType != null && !(dynamicDataSources.containsKey(dataSourceType) || dynamicDataSourcesPublic.containsKey(dataSourceType))){
Log.info("appKey" + dataSourceType + "The corresponding data source configuration was not found, please check the data source configuration");
Throw new RuntimeException("appKey:" + dataSourceType + "The corresponding data source configuration was not found, please check the data source configuration");
}
if(!dynamicDataSources.containsKey(dataSourceType) && dynamicDataSourcesPublic.containsKey(dataSourceType)){
dynamicDataSources.put(dataSourceType, dynamicDataSourcesPublic.get(dataSourceType));
super.afterPropertiesSet();
}
return dataSourceType;
}
public static void setDataSourceType(String dataSourceType) {
contextHolder.set(dataSourceType);
}
public static String getDataSourceType() {
return contextHolder.get();
}
public static void clearDataSourceType() {
contextHolder.remove();
}
@Override
public void setTargetDataSources(Map<Object, Object> targetDataSources) {
dynamicDataSources.putAll(targetDataSources);
super.setTargetDataSources(dynamicDataSources);
super.afterPropertiesSet();
}
}
helloworlde
archerForzmy
Justice-love
CarolineHuang5954
而这个dataSourceCache是个map，但只put了一个resourceId
所以导致的这个map永远只有一个。
不知道我理解的有没有问题
ggndnn
这个resourceId是作为参数传进来的，用来获取dataSourceProxy代理的。。这个参数我看了代码应该是在发起事务回滚的时候传进来的，我是不明白这里这个参数是从哪传来的，应该不是init方法吧
huangjian888
Ⅰ. Issue Description
目前分布式事务应用中使用的是动态数据源，通过对动态数据源A和B的操作发现可以正常插入回滚日志到对应数据源的undo_log表中，但是分布式事务回滚和提交时无法正确路由到对应的数据源对回滚日志和业务数据进行操作
Ⅱ. Describe what happened
场景描述： 应用1:：事务参与方，通过动态数据源连接数据库，有默认数据源 应用2：事务发起方，无数据源配置，调用应用1的接口，通过传参动态对数据源进行操作 数据源A：业务表person 数据源B：业务表user 数据源C：默认数据源，seata-server配置此数据源，global_table, branch_table, lock_table在这
场景1：应用2通过调用两次应用1的dubbo接口进行A和B的数据库操作
场景2：应用2通过调用两次应用1的dubbo接口进行A和B的数据库操作
以下为报C数据源的undo_log找不到的堆栈信息 If there is an exception, please attach the exception trace:
Ⅲ. Describe what you expected to happen
应用启动时通过beanFactory手动注册了全部数据源(从数据源服务中查询来，支持动态添加和更改，没有配置)
SeataAutoConfig.java中通过applicationContext获取dataSource
通过在DAO层添加切面，改变线程变量dataSourceType动态设置数据源
Ⅲ. Seata Version