alibaba / yugong

阿里巴巴去Oracle数据迁移同步工具(全量+增量,目标支持MySQL/DRDS)
GNU General Public License v2.0
2.5k stars 957 forks source link

oracle clob字段无法用yugong做同步 #38

Open CHENsophie opened 7 years ago

CHENsophie commented 7 years ago

您好: 我使用yugong做oracle到oracle的同步时,发现含clob字段的表无法进行数据同步。 以下就是我们运行时候的报错,麻烦帮忙解决下。 [oracle@dell-oracle GDSIGNAL.FLOW_INPUT]$ more table.log 2016-12-28 10:49:10.365 [main] WARN c.t.yugong.extractor.oracle.OracleRecRecordExtractor - mlog[MLOG$_FLOW_INPUT] is exist, just have fun. 2016-12-28 10:49:10.538 [main] INFO c.t.y.e.o.OracleFullRecordExtractor$ContinueExtractor - GDSIGNAL.FLOW_INPUT start postion:9 2016-12-28 10:49:10.540 [main] INFO com.taobao.yugong.controller.YuGongInstance - table[GDSIGNAL.FLOW_INPUT] start successful. extractor:com.taobao.yugong.extractor.oracle.OracleAllRecord Extractor , applier:com.taobao.yugong.applier.AllRecordApplier, translator:NULL 2016-12-28 10:49:12.462 [MultiThreadFullRecordApplier-GDSIGNAL.FLOW_INPUT] INFO o.s.beans.factory.xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/ springframework/jdbc/support/sql-error-codes.xml] 2016-12-28 10:49:12.687 [MultiThreadFullRecordApplier-GDSIGNAL.FLOW_INPUT] INFO org.springframework.jdbc.support.SQLErrorCodesFactory - SQLErrorCodes loaded: [DB2, Derby, H2, HSQL, Inform ix, MS-SQL, MySQL, Oracle, PostgreSQL, Sybase] 2016-12-28 10:49:12.711 [MultiThreadFullRecordApplier-GDSIGNAL.FLOW_INPUT] ERROR com.alibaba.druid.pool.DruidDataSource - recyle error java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219) ~[na:1.7.0_79] at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340) ~[na:1.7.0_79] at com.alibaba.druid.pool.DruidDataSource.recycle(DruidDataSource.java:1297) ~[druid-1.0.17.jar:1.0.17] at com.alibaba.druid.pool.DruidPooledConnection.recycle(DruidPooledConnection.java:297) [druid-1.0.17.jar:1.0.17] at com.alibaba.druid.pool.DruidPooledConnection.close(DruidPooledConnection.java:247) [druid-1.0.17.jar:1.0.17] at org.springframework.jdbc.datasource.DataSourceUtils.doReleaseConnection(DataSourceUtils.java:313) [spring-2.5.6.jar:2.5.6] at org.springframework.jdbc.datasource.DataSourceUtils.releaseConnection(DataSourceUtils.java:274) [spring-2.5.6.jar:2.5.6] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:605) [spring-2.5.6.jar:2.5.6] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:619) [spring-2.5.6.jar:2.5.6] at com.taobao.yugong.applier.FullRecordApplier.applierByBatch(FullRecordApplier.java:103) [yugong-1.0.2.jar:na] at com.taobao.yugong.applier.FullRecordApplier.doApply(FullRecordApplier.java:89) [yugong-1.0.2.jar:na] at com.taobao.yugong.applier.MultiThreadFullRecordApplier$1.run(MultiThreadFullRecordApplier.java:93) [yugong-1.0.2.jar:na] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_79] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_79] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_79] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_79] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] 2016-12-28 10:49:12.711 [YuGongInstance-GDSIGNAL.FLOW_INPUT] ERROR com.taobao.yugong.controller.YuGongInstance - retry 1 ,something error happened. caused by java.lang.RuntimeException: ja va.util.concurrent.ExecutionException: org.springframework.jdbc.UncategorizedSQLException: PreparedStatementCallback; uncategorized SQLException for SQL [insert into GDSIGNAL.FLOW_INPUT(PA TH , FLOW_EXEC_ID , INPUTINDEX , INPUTSIZE , NAME , CDO_NAME , RECORDS , CHANNEL_KEY , ID) values (? , ? , ? , ? , ? , ? , ? , ? , ?)]; SQL state [null]; error code [0]; Error; nested exception is java.sql.SQLException: Error at com.taobao.yugong.common.utils.thread.ExecutorTemplate.waitForResult(ExecutorTemplate.java:79) at com.taobao.yugong.applier.MultiThreadFullRecordApplier.apply(MultiThreadFullRecordApplier.java:103) at com.taobao.yugong.applier.AllRecordApplier.apply(AllRecordApplier.java:58) at com.taobao.yugong.controller.YuGongInstance$1.processTable(YuGongInstance.java:215) at com.taobao.yugong.controller.YuGongInstance$1.run(YuGongInstance.java:145) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.concurrent.ExecutionException: org.springframework.jdbc.UncategorizedSQLException: PreparedStatementCallback; uncategorized SQLException for SQL [insert into GDSIGNAL. FLOW_INPUT(PATH , FLOW_EXEC_ID , INPUTINDEX , INPUTSIZE , NAME , CDO_NAME , RECORDS , CHANNEL_KEY , ID) values (? , ? , ? , ? , ? , ? , ? , ? , ?)]; SQL state [null]; error code [0]; Error; nested exception is java.sql.SQLException: Error at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:188) at com.taobao.yugong.common.utils.thread.ExecutorTemplate.waitForResult(ExecutorTemplate.java:77) ... 5 more Caused by: org.springframework.jdbc.UncategorizedSQLException: PreparedStatementCallback; uncategorized SQLException for SQL [insert into GDSIGNAL.FLOW_INPUT(PATH , FLOW_EXEC_ID , INPUTIND EX , INPUTSIZE , NAME , CDO_NAME , RECORDS , CHANNEL_KEY , ID) values (? , ? , ? , ? , ? , ? , ? , ? , ?)]; SQL state [null]; error code [0]; Error; nested exception is java.sql.SQLException: Error at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:83) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:607) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:619) at com.taobao.yugong.applier.FullRecordApplier.applyOneByOne(FullRecordApplier.java:145) at com.taobao.yugong.applier.FullRecordApplier.applierByBatch(FullRecordApplier.java:136) at com.taobao.yugong.applier.FullRecordApplier.doApply(FullRecordApplier.java:89) at com.taobao.yugong.applier.MultiThreadFullRecordApplier$1.run(MultiThreadFullRecordApplier.java:93) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ... 1 more Caused by: java.sql.SQLException: Error at com.alibaba.druid.pool.DruidDataSource.handleConnectionException(DruidDataSource.java:1211) at com.alibaba.druid.pool.DruidPooledConnection.handleException(DruidPooledConnection.java:132) at com.alibaba.druid.pool.DruidPooledStatement.checkException(DruidPooledStatement.java:68) at com.alibaba.druid.pool.DruidPooledPreparedStatement.setObject(DruidPooledPreparedStatement.java:467) at com.taobao.yugong.applier.FullRecordApplier$3.doInPreparedStatement(FullRecordApplier.java:153) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:591) ... 12 more Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to oracle.sql.CLOB at oracle.jdbc.driver.OraclePreparedStatement.setObjectCritical(OraclePreparedStatement.java:9198) at oracle.jdbc.driver.OraclePreparedStatement.setObjectInternal(OraclePreparedStatement.java:8812) at oracle.jdbc.driver.OraclePreparedStatement.setObject(OraclePreparedStatement.java:9285) at com.alibaba.druid.pool.DruidPooledPreparedStatement.setObject(DruidPooledPreparedStatement.java:465) ... 14 more

agapple commented 7 years ago

源库和目标库的建表语句发一下,我关注一下数据库类型

CHENsophie commented 7 years ago

源库和目标库都是一样的建表语句: -- Create table create table CM_CARPO_FLOW_INPUT ( id NUMBER not null, flow_exec_id VARCHAR2(32), channel_key VARCHAR2(255), name VARCHAR2(255), cdo_name VARCHAR2(255), path CLOB, inputsize NUMBER, records NUMBER, inputindex NUMBER(10) ) tablespace ROE_DW_FTAB_NEW_01 pctfree 10 initrans 1 maxtrans 255 storage ( initial 8M next 8M minextents 1 maxextents unlimited pctincrease 0 ); alter table CM_CARPO_FLOW_INPUT add constraint PK_CM_CARPO_FLOW_INPUT primary key (ID) using index tablespace ROE_CARPO_CONF pctfree 10 initrans 2 maxtrans 255 storage ( initial 64K next 1M minextents 1 maxextents unlimited pctincrease 0 );

haozi4go commented 6 years ago

请问这个问题是如何解决的呢,求解答