alibaba / DataX

DataX是阿里云DataWorks数据集成的开源版本。
Other
15.83k stars 5.4k forks source link

DataX抽取数据,数据源MongoDB字段值为null抽取报错 #2143

Open 18616860694 opened 3 months ago

18616860694 commented 3 months ago

【版本号】 DataX:1.5

【描述】 MongoDB作为数据源写入Oracle,当脚本读取MongoDB的字段值为null时,会报源端与目标端字段不匹配。

【报错信息】

2024-06-25 09:46:14 [AnalysisStatistics.analysisStatisticsLog-53] Exception in thread "taskGroup-0" com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-00], Description:[您的配置错误.].  - 列配置信息有错误. 因为您配置的任务中,源头读取字段数:1 与 目的表要写入的字段数:2 不相等. 请检查您的配置并作出修改.
2024-06-25 09:46:14 [AnalysisStatistics.analysisStatisticsLog-53]   at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:26)
2024-06-25 09:46:14 [AnalysisStatistics.analysisStatisticsLog-53]   at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:278)
2024-06-25 09:46:14 [AnalysisStatistics.analysisStatisticsLog-53]   at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:319)
2024-06-25 09:46:14 [AnalysisStatistics.analysisStatisticsLog-53]   at com.alibaba.datax.plugin.writer.oraclewriter.OracleWriter$Task.startWrite(OracleWriter.java:88)
2024-06-25 09:46:14 [AnalysisStatistics.analysisStatisticsLog-53]   at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)
2024-06-25 09:46:14 [AnalysisStatistics.analysisStatisticsLog-53]   at java.lang.Thread.run(Thread.java:748)

【脚本如下】

{
  "job": {
    "setting": {
      "speed": {
        "channel": 3,
        "byte": 1048576
      },
      "errorLimit": {
        "record": 0,
        "percentage": 0.02
      }
    },
    "content": [
      {
        "reader": {
          "name": "mongodbreader",
          "parameter": {
            "address": [
              "xxx"
            ],
            "userName": "xxx",
            "userPassword": "xxx",
            "dbName": "cc_logs",
            "collectionName": "xxx",
            "column": [
              {
                "name": "_id",
                "type": "Object"
              },
              {
                "name": "ACCESS_NUMBER_DESC",
                "type": "String"
              }
            ]
          }
        },
        "writer": {
          "name": "oraclewriter",
          "parameter": {
            "username": "xxx",
            "password": "xxx",
            "column": [
              "ID",
              "ACCESS_NUMBER_DESC"
            ],
            "connection": [
              {
                "table": [
                  "xxx"
                ],
                "jdbcUrl": "jdbc:oracle:thin:@//xxx:xxx/xxx "
              }
            ]
          }
        }
      }
    ]
  }
}

【备注】 源端与目标端均可正常连接,且自测当MongoDB字段【ACCESS_NUMBER_DESC】不为NULL时,脚本可正常插入,一旦【ACCESS_NUMBER_DESC】中有值为NULL时就报错。 不知道和版本有没有关系?也尝试了用query的方法,但是好像没办法读取MongoDB的sql,不知道有没有大佬有遇到过类似的问题,是怎么解决的?还望不吝赐教,感谢!