datavane / tis

Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI
https://tis.pub
Apache License 2.0
1.04k stars 221 forks source link

达梦数据库最新版本有找不到表的情况 #354

Closed baisui1981 closed 2 months ago

baisui1981 commented 2 months ago

测试场景

批量同步异常:

Caused by: java.lang.RuntimeException: TableNotFound{ dbId=merchant,tableName='ad_company_bak,url:jdbc:dm://192.168.5.121:30236?schema=merchant'}
    at com.qlangtech.tis.plugin.ds.BasicDataSourceFactory.getTableMetadata(BasicDataSourceFactory.java:124)
    at com.alibaba.datax.plugin.rdbms.util.DBUtil.getTableColums(DBUtil.java:527)
    at com.alibaba.datax.plugin.rdbms.util.DBUtil.getTableColumnsByConn(DBUtil.java:535)
    at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.dealColumnConf(OriginalConfPretreatmentUtil.java:108)
    at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.dealColumnConf(OriginalConfPretreatmentUtil.java:151)
    at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:37)
    at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Job.init(CommonRdbmsWriter.java:51)
    at com.alibaba.datax.plugin.dameng.writer.DaMengWriter$Job.init(DaMengWriter.java:40)
    at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:665)
    at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:319)
    at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:148)
    ... 7 common frames omitted
Caused by: com.qlangtech.tis.plugin.ds.TableNotFoundException: null
    at com.qlangtech.tis.plugin.ds.DataSourceFactory.parseTableColMeta(DataSourceFactory.java:213)
    at com.qlangtech.tis.plugin.ds.DataSourceFactory.lambda$parseTableColMeta$0(DataSourceFactory.java:345)
    at com.qlangtech.tis.plugin.ds.DataSourceFactory.validateConnection(DataSourceFactory.java:128)
    at com.qlangtech.tis.plugin.ds.DataSourceFactory.parseTableColMeta(DataSourceFactory.java:344)
    at com.qlangtech.tis.plugin.ds.BasicDataSourceFactory.parseTableColMeta(BasicDataSourceFactory.java:144)
    at com.qlangtech.tis.plugin.ds.BasicDataSourceFactory.lambda$getTableMetadata$1(BasicDataSourceFactory.java:114)
    at com.qlangtech.tis.plugin.ds.DBConfig.vistDbName(DBConfig.java:130)
    at com.qlangtech.tis.plugin.ds.BasicDataSourceFactory.getTableMetadata(BasicDataSourceFactory.java:113)
    ... 17 common frames omitted
baisui1981 commented 2 months ago

66231725703602_ pic 由于用户填写的 dbName可能和 真实的dbName大小写不一致,导致在取表的时候由此,jdbc接口上通过获取metadata取不到表的情况。 所以,需要在datasource中加一个schema 小小写是否一致的校验证。